A user study of auditory, head-up and multi-modal displays in vehicles

Impacts of information quantity and display formats on driving behaviors in a connected vehicle environment

The emerging connected vehicle (CV) technologies facilitate the development of integrated advanced driver assistance systems (ADASs), with which various functions are coordinated in a comprehensive framework. However, challenges arise in enabling drivers to perceive important information with minimal distractions when multiple messages are simultaneously provided by integrated ADASs. To this end, this study introduces three types of human-machine interfaces (HMIs) for an integrated ADAS: 1) three messages using a visual display only, 2) four messages using a visual display only, and 3) three messages using visual plus auditory displays. Meanwhile, the differences in driving performance across three HMI types are examined to investigate the impacts of information quantity and display formats on driving behaviors. Additionally, variations in drivers' responses to the three HMI types are examined. Driving behaviors of 51 drivers with respect to three HMI types are investigated in eight field testing scenarios. These scenarios include warnings for rear-end collision, lateral collision, forward collision, lane-change, and curve speed, as well as notifications for emergency events downstream, the specified speed limit, and car-following behaviors. Results indicate that, compared to a visual display only, presenting three messages through visual and auditory displays enhances driving performance in four typical scenarios. Compared to the presentation of three messages, a visual display offering four messages improves driving performance in rear-end collision warning scenarios but diminishes the performance in lane-change scenarios. Additionally, the relationship between information quantity and display formats shown on HMIs and driving performance can be moderated by drivers' gender, occupation, driving experience, annual driving distance, and safety attitudes. Findings are indicative to designers in automotive industries in developing HMIs for future CVs.

Compact pupil-expansion AR-HUD based on surface-relief grating

Augmented reality head-up display (AR-HUD) using diffractive waveguide is a challenging research field. It can drastically reduce the system volume compared with AR-HUD based on freeform mirror. However, one of the remaining challenges that affects the performance of the diffractive waveguide is to expand the eye-box while maintaining the illuminance uniformity. In this paper, a one-dimensional pupil expansion diffractive optical waveguide system for AR-HUD is presented. The optimization of grating parameters is based on scalar diffraction theory and rigorous coupled wave analysis (RCWA). Then, the illuminance uniformity is optimized through non-sequential ray tracing. We simulate and construct a waveguide-based AR-HUD. The presented AR-HUD realized an exit pupil size of 80 mm × 15 mm and a field of view of 10° × 5° at the wavelength of 532 nm.

Effects of collision warning characteristics on driving behaviors and safety in connected vehicle environments

With the emerging connected vehicle (CV) technologies, a novel in-vehicle omni-direction collision warning system (OCWS) is developed. For example, vehicles approaching from different directions can be detected, and advanced collision warnings caused by vehicles approaching from different directions can be provided. Effectiveness of OCWS in reducing crash and injury related to forward, rear-end and lateral collision is recognized. However, it is rare that the effects of collision warning characteristics including collision types and warning types on micro-level driver behaviors and safety performance is assessed. In this study, variations in drivers' responses among different collision types and between visual only and visual plus auditory warnings are examined. In addition, moderating effects by driver characteristics including drivers' demographics, years of driving experience, and annual driving distance are also considered. An in-vehicle human-machine interface (HMI) that can provide both visual and auditory warnings for forward, rear-end, and lateral collisions is installed on an instrumented vehicle. 51 drivers participate in the field tests. Performance indicators including relative speed change, time taken to accelerate/decelerate, and maximum lateral displacement are adopted to reflect drivers' responses to collision warnings. Then, generalized estimation equation (GEE) approach is applied to examine the effects of drivers' characteristics, collision type, warning type and their interaction on the driving performance. Results indicate that age, year of driving experience, collision type, and warning type can affect the driving performance. Findings should be indicative to the optimal design of in-vehicle HMI and thresholds for the activation of collision warnings that can increase the drivers' awareness to collision warnings from different directions. Also, implementation of HMI can be customized with respect to individual driver characteristics.

Input modality matters: A comparison of touch, speech, and gesture based in-vehicle interaction

Innovative input devices are being available for in-vehicle information systems (IVISs). While they have the potential to provide enjoyable driving by enabling drivers to perform non-driving related tasks (NDRTs) in more natural ways, the associated distracting effects should be paid with more attention. The purpose of this exploratory study was to compare the effects of three novel input modalities, i.e., touchscreen-based interaction (TBI), speech-based interaction (SBI), and gesture-based interaction (GBI), on driving performance and driver visual behaviors. Moreover, we examined if the influence of different modalities would be moderated by the difficulty level of NDRTs. A total of 36 participants were invited to a simulated driving experiment where they were randomly assigned to one of the four groups (TBI, GBI, SBI or baseline) and completed three driving trials. The results showed that TBI led to the worse driving performance, as indicated by the significantly prolonged reaction time, reduced minimum time-to-collision, and increased variations in both longitudinal and lateral vehicle control. The deteriorated driving performance could be attributed, at least partially, to the intense visual demand induced by looking towards the touchscreen, as indicated by more and longer off-the-road glances. The adverse impacts of GBI were relatively smaller, but it still posed great crash risk by leading to a shorter minimum time-to-collision and less stable vehicle control compared to the baseline. SBI, although not completely equivalent to the baseline group, showed the minimum influence on driving and visual performance. Only very few interaction effects were found, suggesting that the effects of modality were quite robust across different NDRTs. It was concluded that SBI and GBI provided safer alternatives to in-vehicle interaction than TBI.

A field study of mental workload: conventional bus drivers versus bus rapid transit drivers

Road traffic accidents are increasing worldwide and cause a high number of fatalities and injuries. Mental Work Load (MWL) is a contributing factor in road safety. The primary aim of this work was to study important MWL factors and then compare conventional and BRT (Bus Rapid Transit) drivers' MWL. This study evaluated bus drivers' MWL using the Driving Activity Load Index (DALI) questionnaire conducted with 123 bus drivers in Tehran. The results revealed significant differences between conventional and BRT drivers' mental workload. Moreover, data modelling showed that some organisational and environmental factors such as bus type, working hours per day, road maze, and route traffic volume contribute to drivers' mental workload. These findings suggest some essential customised factors that may help measure and offer practical solutions for decreasing the level of bus drivers' MWL in real-world road driving. Practitioner summaryMental workload is affected by several contributing factors. Depending on the working context, some of these contributing factors have a more significant influence on the level of the experienced MWL. Therefore, the main factors influencing the MWL of BRT and conventional bus drivers were assessed in their real-life environment.Abbreviations: MWL: mental work load; BRT: bus rapid transit; CB: conventional bus; DALI: driving activity load index; NASA-TLX: NASA task load index; SWAT: subjective workload assessment technique; EEG: electroencephalography electrocardiogram; fNIRS: functional magnetic resonance imaging; ITS: intelligent transportation systems; AVL: automated vehicle location.

Automotive Holographic Head-Up Displays

Driver's access to information about navigation and vehicle data through in-car displays and personal devices distract the driver from safe vehicle management. The discrepancy between road safety and infotainment must be addressed to develop safely operated modern vehicles. Head-up displays (HUDs) aim to introduce a seamless uptake of visual information for the driver while securely operating a vehicle. HUDs projected on the windshield provide the driver with visual navigation and vehicle data within the comfort of the driver's personal eye box through a customizable extended display space. Windshield HUDs do not require the driver to shift the gaze away from the road to attain road information. This article presents a review of technological advances and future perspectives in holographic HUDs by analyzing the optoelectronics devices and the user experience of the driver. The review elucidates holographic displays and full augmented reality in 3D with depth perception when projecting the visual information on the road within the driver's gaze. Design factors, functionality, and the integration of personalized machine learning technologies into holographic HUDs are discussed. Application examples of the display technologies regarding road safety and security are presented. An outlook is provided to reflect on display trends and autonomous driving.

Attention Pedestrians Ahead: Evaluating User Acceptance and Perceptions of a Cooperative Intelligent Transportation System-Warning System for Pedestrians

Warning system for pedestrians (WSP), one of cooperative intelligent transport system (C-ITS) applications, is designed to increase safety for pedestrians but also for drivers and other road users. The evaluation of end-user acceptance and perceptions of this technology is crucial before deploying it in transportation systems. Five WSP human–machine interfaces (HMIs) were designed and simulated using a driver’s first-view video footage of driving through a pedestrian crossing in Newcastle upon Tyne. The five WSP designs were evaluated with 24 younger end users (35 years old and younger). This study first evaluated the usefulness of the unified theory of acceptance and use of technology (UTAUT) in modelling end-user acceptance in terms of behavioural intentions to use WSP. The results suggest that the UTAUT can be applied to investigate the end-user acceptance of WSP, with performance expectancy and effort expectancy influencing the behavioural intentions to use WSP. Furthermore, we investigated end-user attitudes towards various WSP human–machine interface (HMI) designs. Participants showed more positive attitudes towards visual-only interfaces than towards audio-only and multi-modal (combinations of visual and audio) interfaces. Above all, the findings of this research increase our understanding of public acceptance and perceptions of this C-ITS application.

Effects of full windshield head-up display on visual attention allocation

This study explores the impact of full windshield head-up display (FHUD) cues on the visual attention allocation of drivers under different scenarios. Forty-eight participants with driving experience were randomly divided into two groups and asked to drive on a preset route in various virtual driving scenarios created in advance. The full windshield highlights situational cues related to the driving task, such as lane lines, safe vehicle distance warnings, navigation guidance and pedestrian cues. Regarding the perception of situational cues, the number of fixations and mean fixation duration with FHUD were lower than those without FHUD. Furthermore, the dwell time percentage of the driver's forward view with FHUD was larger than that without FHUD, and the dwell time percentage on both sides was smaller than that without FHUD. In conclusion, FHUD may help drivers more effectively perceive cues and improve drivers' visual attention allocation. Practitioner summary: FHUD may affect drivers' attention while driving. We examined the effect of FHUD on number of fixations, fixation duration and dwell time percentage in the area of interest under different weather scenarios. Experimental results indicated that FHUD could improve drivers' visual attention allocation and help drivers more effectively perceive cues. Abbreviations: FHUD: full windshield head-up display; HUD: head-up display; HDD: head-down display; AOI: area of interests; AR: augmented reality.

Effects of Mobile Computer Terminal Configuration and Level of Driving Control on Police Officers' Performance and Workload

OBJECTIVE

The objective of this study was to assess police officers' performance and workload in using two mobile computer terminal (MCT) configurations under operational and tactical driving conditions.

BACKGROUND

Crash reports have identified in-vehicle distraction to be a major cause of law enforcement vehicle crashes. The MCT has been found to be the most frequently used in-vehicle technology and the main source of police in-vehicle distraction.

METHOD

Twenty police officers participated in a driving simulator-based assessment of driving behavior, task completion time, and perceived workload with two MCT configurations under operational and tactical levels of driving.

RESULTS

The findings revealed that using the MCT configuration with speech-based data entry and head-up display location while driving improved driving performance, decreased task completion time, and reduced police officers' workload as compared to the current MCT configuration used by police departments. Officers had better driving but worse secondary task performance under the operational driving as compared to the tactical driving condition.

CONCLUSION

This study provided an empirical support for use of an enhanced MCT configuration in police vehicles to improve police officers' safety and performance. In addition, the findings emphasize the need for more training to improve officers' tactical driving skills and multitasking behavior.

APPLICATION

The findings provide guidelines for vehicle manufacturers, MCT developers, and police agencies to improve the design and implementation of MCTs in police vehicles considering input modality and display eccentricity, which are expected to increase officer and civilian safety.

A Systematic Review of Virtual Reality Applications for Automated Driving: 2009–2020

While virtual reality (VR) interfaces have been researched extensively over the last decades, studies on their application in vehicles have only recently advanced. In this paper, we systematically review 12 years of VR research in the context of automated driving (AD), from 2009 to 2020. Due to the multitude of possibilities for studies with regard to VR technology, at present, the pool of findings is heterogeneous and non-transparent. We investigated N = 176 scientific papers of relevant journals and conferences with the goal to analyze the status quo of existing VR studies in AD, and to classify the related literature into application areas. We provide insights into the utilization of VR technology which is applicable at specific level of vehicle automation and for different users (drivers, passengers, pedestrians) and tasks. Results show that most studies focused on designing automotive experiences in VR, safety aspects, and vulnerable road users. Trust, simulator and motion sickness, and external human-machine interfaces (eHMIs) also marked a significant portion of the published papers, however a wide range of different parameters was investigated by researchers. Finally, we discuss a set of open challenges, and give recommendation for future research in automated driving at the VR side of the reality-virtuality continuum.

Diffuser screen with flat-top angular distribution of scattered light realized by a dual-beam holographic speckle

Holographic speckle screens with the Gaussian type distribution of scattered light, which are used to increase the viewing angle of the image in projection display systems, result in nonuniform image brightness in different observing positions. In this study, based on Helmholtz-Kirchhoff theory, a dual-beam scattering theory of rough surface is derived. By analyzing the spatial frequency spectrum of the scattered light, it is found that when two laser beams irradiated the ground glass at a certain angle, the resulting speckles recorded on the photoresist can generate a flat-top angular distribution of the scattered light. Speckle screens are fabricated by two light beams at different angles, and the angular intensity distribution of scattered light is measured. The results are in good agreement with the theory. Compared with the Gaussian type diffuser, the energy efficiency of the speckle screen proposed has a 46% increase when the angular luminance uniformity is set to be 80%, which effectively improves the brightness when used in a head up display system.

Assessment of Drivers' Perceptions of Connected Vehicle-Human Machine Interface for Driving Under Adverse Weather Conditions: Preliminary Findings From Wyoming

Connected vehicle (CV) technology aims to improve drivers' situational awareness through audible and visual warnings displayed on a human-machine interface (HMI), thus reducing crashes caused by human error. This paper developed a driving simulator test bed to assess the readability and usefulness of the Wyoming CV applications. A total number of 26 professional drivers were recruited to participate in a driving-simulator study. Prior to driving the simulator, the participants were trained on both the concept of CV technology and the developed CV applications as well as the operation of the driving simulator. Three driving simulation scenarios were designed. For each scenario, participants drove two times: one with the HMI turned on and another one with the HMI turned off. After driving the simulator, a comprehensive revealed-preference survey was employed to collect the participants' perceptions of CV technology and Wyoming CV applications. Results show that the Wyoming CV applications were most favored under poor-visibility driving conditions. Among the Wyoming CV applications, forward collision warning and rerouting applications were experienced as the most useful. Approximately 89% of the participants stated that the Wyoming CV applications provided them with improved road condition information and increased their experienced safety while driving; 65% of the participants stated the CV applications and the HMI did not introduce distraction from the primary task of driving. Finally, this paper concludes that the design of CV HMI needs to balance a trade-off between the readability of the warnings and drivers' capability to safely recognize and timely respond to the received warnings.

Effects of speech-based intervention with positive comments on reduction of driver's anger state and perceived workload, and improvement of driving performance

Research suggests that speech-based interventions can mitigate driving anger and enhance road safety. The present study found that both positive and negative comments can reduce anger state and perceived workload, and improve driving performance. In addition, positive comment including description of the driving environment and comment on drivers is more effective than negative comment intervention, which is indicated by larger effect size and higher user satisfaction and acceptance. The research findings could provide practical implications on the design of in-vehicle intelligent agents for driving behavior intervention.

Comparison of in-sight and handheld navigation devices toward supporting industry 4.0 supply chains: First and last mile deliveries at the human level

Last (and First) mile deliveries are an increasingly important and costly component of supply chains especially those that require transport within city centres. With reduction in anticipated manufacturing and delivery timescales, logistics personnel are expected to identify the correct location (accurately) and supply the goods in appropriate condition (safe delivery). Moving towards more environmentally sustainable supply chains, the last/first mile of deliveries may be completed by a cyclist courier which could result in significant reductions in congestion and emissions in cities. In addition, the last metres of an increasing number of deliveries are completed on foot i.e. as a pedestrian. Although research into new technologies to support enhanced navigation capabilities is ongoing, the focus to date has been on technical implementations with limited studies addressing how information is perceived and actioned by a human courier. In the research reported in this paper a comparison study has been conducted with 24 participants evaluating two examples of state-of-the-art navigation aids to support accurate (right time and place) and safe (right condition) navigation. Participants completed 4 navigation tasks, 2 whilst cycling and 2 whilst walking. The navigation devices under investigation were a handheld display presenting a map and instructions and an in-sight monocular display presenting text and arrow instructions. Navigation was conducted in a real-world environment in which eye movements and device interaction were recorded using Tobii-Pro 2 eye tracking glasses. The results indicate that the handheld device provided better support for accurate navigation (right time and place), with longer but less frequent gaze interactions and higher perceived usability. The in-sight display supported improved situation awareness with a greater number of hazards acknowledged. The benefits and drawbacks of each device and use of visual navigation support tools are discussed.

Claudel C. Evaluating Signalization and Channelization Selections at Intersections Based on an Entropy Method

Direct left turns (DLTs) could cause traffic slowdown, delay, stops, and even accidents on intersections, especially on no-median roads. Channelization and signalization can significantly diminish negative impact of DLTs. In China, a total of 56 large and medium-sized cities, including 17 provincial capitals, have adopted vehicle restriction policies due to traffic congestion, vehicle energy conservation and emission reduction, which cause travel inconvenience for citizens. This paper mainly studies signalization and channelization selections at intersections based on an entropy method. Based on the commonly used three evaluation indexes, the number of vehicles, CO emissions and fuel consumption have been added. The entropy evaluation method (EEM) method is innovatively used to objectively calculate the weight of the six indexes, which carry out the optimal traffic volume combinations for intersections of present situation, channelization and signalization. A VISSIM simulation is also used to evaluate the operating status of three conditions. The results show that EEM could help enormously in choosing different methods at a certain intersection. With the EEM, six indexes decrease by 20–70% at most.

Simulation-based evaluation of an in-vehicle smart situation awareness enhancement system

Situation awareness (SA) constitutes a critical factor in road safety, strongly related to accidents. This paper describes the evaluation of a proposed SA enhancement system (SAES) that exploits augmented reality through a head-up display (HUD). Two SAES designs were evaluation (information rich vs. minimal information) using a custom-made simulator and the Situation Awareness Global Assessment Technique with performance and EEG measures. The paper describes the process of assessing the SA of drivers using the SAES, through a series of experiments with participants in a Cave Automatic Virtual Environment. The effectiveness of the SAES was tested in a within-group research design. The results showed that the information rich (radar-style display) was superior to the minimal (arrow hazard indicator) design and that both SAES improved drivers' SA and performance compared to the control (no HUD) design. Practitioner Summary: Even though driver situation awareness is considered as one of the leading causes of road accidents, little has been done to enhance it. The current study demonstrates the positive effect of a proposed situation awareness enhancement system on driver situation awareness, through an experiment using virtual prototyping in a simulator.

Digital, analogue, or redundant speedometers for truck driving: Impact on visual distraction, efficiency and usability

Existing literature does not draw conclusions as to which speedometer type is better for truck driving. A digital speedometer would be more beneficial when obtaining absolute and relative readings, while an analogue speedometer would be more efficient and less distracting when detecting dynamic speed changes. Redundant speedometers, which simultaneously present digital and analogue speedometers, appear increasingly in vehicles, but no information is available on their ergonomic qualities. This study compared three speedometers: digital speedometers, analogue speedometers, and redundant speedometers. This study compared the efficiency, usability and visual distraction measures for all three types of speedometers in a simulated truck driving setting. The task-dependant results were confirmed for the digital and analogue speedometer. The redundant speedometer combined the benefits of each type presented separately, which highlights interesting theoretical and applied implications.

Sensitivity evaluation of the visual, tactile, and auditory detection response task method while driving

OBJECTIVES

In this article, we evaluate the sensitivity to cognitive load of 3 versions of the Detection Response Task method (DRT), proposed in ISO Draft Standard DIS-17488.

METHODS

We present a user study with 30 participants in which we compared the sensitivity to cognitive load of visual, audio, and tactile DRT in a simulated driving environment. The amount of cognitive load was manipulated with secondary n-back tasks at 2 levels of difficulty (0-back and 1-back). We also explored whether the DRT method is least sensitive to cognitive load when the stimuli and secondary task are of the same modality. For this purpose, we used 3 forms to present the n-back task stimuli: visual, audio, and tactile. Responses to the task were always vocal. The experiment was based on a between-subject design (the DRT modalities) with 2 levels of within-subject design study (modalities and difficulty of the secondary n-back tasks). The participants' primary task in the study was to drive safely, and a second priority was to answer to DRT stimuli and perform secondary tasks.

RESULTS

The results indicate that all 3 versions of the DRT tested were sensitive to detecting the difference in cognitive load between the reference driving period and driving and engaging in the secondary tasks. Only the visual DRT discriminated between the 0-back and 1-back conditions on mean response time. Contrary to expectations, no interaction was observed between DRT modality and the stimuli modality used for presentation of the secondary tasks.

CONCLUSIONS

None of the 3 methods of presenting DRT stimuli showed a consistent advantage in sensitivity in differentiating multiple levels of cognitive load if all response times, hit rates, and secondary task performance are considered. If only response time is considered, the visual presentation of the DRT stimulus used in this study showed some advantages. In interpreting these data, it should be noted that the methods of DRT stimulus presentation varied somewhat from the currently proposed draft ISO standard and it is possible that the relative salience level of the visual DRT stimulus influenced the findings. It is further suggested that more than 2 levels of difficulty of the n-back task should be considered for further investigation of the relative sensitivity of different DRT stimuli modalities. Parameters that indicate change in cognitive load (response time, hit rate, task performance) should be analyzed together in assessing the overall impact on the driver and not individually, in order to obtain a fuller insight of the assessed cognitive load.

Extending helicopter operations to meet future integrated transportation needs

Helicopters have the potential to be an integral part of the future transport system. They offer a means of rapid transit in an overly populated transport environment. However, one of the biggest limitations on rotary wing flight is their inability to fly in degraded visual conditions in the critical phases of approach and landing. This paper presents a study that developed and evaluated a Head up Display (HUD) to assist rotary wing pilots by extending landing to degraded visual conditions. The HUD was developed with the assistance of the Cognitive Work Analysis method as an approach for analysing the cognitive work of landing the helicopter. The HUD was tested in a fixed based flight simulator with qualified helicopter pilots. A qualitative analysis to assess situation awareness and workload found that the HUD enabled safe landing in degraded conditions whilst simultaneously enhancing situation awareness and reducing workload. Continued development in this area has the potential to extend the operational capability of helicopters in the future.