SPIDER: A Framework for Understanding Driver Distraction

The Effects of Auditory Working Memory Task on Situation Awareness in Complex Dynamic Environments: An Eye-movement Study

OBJECTIVE

This study investigated the effect of auditory working memory task on situation awareness (SA) and eye-movement patterns in complex dynamic environments.

BACKGROUND

Many human errors in aviation are caused by a lack of SA, and distraction from auditory secondary tasks is a serious threat to SA. However, it remains unclear how auditory working memory tasks affect SA and eye-movement patterns.

METHOD

Participants (n = 28) were randomly allocated to two groups and received different periods of visual search training (short versus long). They subsequently completed a situation awareness measurement task in three auditory secondary task conditions (without secondary task, auditory calculation task, and auditory 2-back task). Eye-movement data were collected during the situation awareness measurement task.

RESULTS

The auditory 2-back task significantly reduced overall SA, Level 1 SA, dwell times, and total percentage of fixation time on task-related areas of interests in the SA measurement task. Overall SA and Level 3 SA were not reduced by the auditory 2-back task in individuals in the longer visual search training time condition.

CONCLUSION

Auditory working memory load impairs SA in the perception and projection stage; however, greater experience can overcome impairment of SA in the projection stage.

APPLICATION

This study provided possible approaches to preventing loss of SA: (1) improving crew members' communication skills to ensure the accurate and clear transmission of information, reducing the difficulty of processing information, and (2) providing targeted cognitive training tailored to each pilot's level of experience.

Modeling distracted driving behavior considering cognitive processes

The modeling of distracted driving behavior has been studied for many years, however, there remain many distraction phenomena that can not be fully modeled. This study proposes a new method that establishes the model using the queuing network model human processor (QN-MHP) framework. Unlike previous models that only consider distracted-driving-related human factors from a mathematical perspective, the proposed method reflects the information processing in the human brain, and simulates the distracted driver's cognitive processes based on a model structure supported by physiological and cognitive research evidence. Firstly, a cumulative activation effect model for external stimuli is adopted to mimic the phenomenon that a driver responds only to stimuli above a certain threshold. Then, dual-task queuing and switching mechanisms are modeled to reflect the cognitive resource allocation under distraction. Finally, the driver's action is modeled by the Intelligent Driver Model (IDM). The model is developed for visual distraction auditory distraction separately. 773 distracted car-following events from the Shanghai Naturalistic Driving Study data were used to calibrate and verify the model. Results show that the model parameters are more uniform and reasonable. Meanwhile, the model accuracy has improved by 57% and 66% compared to the two baseline models respectively. Moreover, the model demonstrates its ability to generate critical pre-crash scenarios and estimate the crash rate of distracted driving. The proposed model is expected to contribute to safety research regarding new vehicle technologies and traffic safety analysis.

On the impact of on-road partially-automated driving on drivers' cognitive workload and attention allocation

The use of partially-automated or SAE level-2 vehicles is expected to change the role of the human driver from operator to supervisor, which may have an effect on the driver's workload and visual attention. In this study, 30 Ontario drivers operated a vehicle in manual and partially-automated mode. Cognitive workload was measured by means of the Detection Response Task, and visual attention was measured by means of coding glances on and off the forward roadway. No difference in cognitive workload was found between driving modes. However, drivers spent less time glancing at the forward roadway, and more time glancing at the vehicle's touchscreen. These data add to our knowledge of how vehicle automation affects cognitive workload and attention allocation, and show potential safety risks associated with the adoption of partially-automated driving.

Distraction "Hangover": Characterization of the Delayed Return to Baseline Driving Risk After Distracting Behaviors

OBJECTIVE

We measured how long distraction by a smartphone affects simulated driving behaviors after the tasks are completed (i.e., the distraction hangover).

BACKGROUND

Most drivers know that smartphones distract. Trying to limit distraction, drivers can use hands-free devices, where they only briefly glance at the smartphone. However, the cognitive cost of switching tasks from driving to communicating and back to driving adds an underappreciated, potentially long period to the total distraction time.

METHOD

Ninety-seven 21- to 78-year-old individuals who self-identified as active drivers and smartphone users engaged in a simulated driving scenario that included smartphone distractions. Peripheral-cue and car-following tasks were used to assess driving behavior, along with synchronized eye tracking.

RESULTS

The participants' lateral speed was larger than baseline for 15 s after the end of a voice distraction and for up to 25 s after a text distraction. Correct identification of peripheral cues dropped about 5% per decade of age, and participants from the 71+ age group missed seeing about 50% of peripheral cues within 4 s of the distraction. During distraction, coherence with the lead car in a following task dropped from 0.54 to 0.045, and seven participants rear-ended the lead car. Breadth of scanning contracted by 50% after distraction.

CONCLUSION

Simulated driving performance drops dramatically after smartphone distraction for all ages and for both voice and texting.

APPLICATION

Public education should include the dangers of any smartphone use during driving, including hands-free.

Let Complexity Bring Clarity: A Multidimensional Assessment of Cognitive Load Using Physiological Measures

The effects of cognitive load on driver behavior and traffic safety are unclear and in need of further investigation. Reliable measures of cognitive load for use in research and, subsequently, in the development and implementation of driver monitoring systems are therefore sought. Physiological measures are of interest since they can provide continuous recordings of driver state. Currently, however, a few issues related to their use in this context are not usually taken into consideration, despite being well-known. First, cognitive load is a multidimensional construct consisting of many mental responses (cognitive load components) to added task demand. Yet, researchers treat it as unidimensional. Second, cognitive load does not occur in isolation; rather, it is part of a complex response to task demands in a specific operational setting. Third, physiological measures typically correlate with more than one mental state, limiting the inferences that can be made from them individually. We suggest that acknowledging these issues and studying multiple mental responses using multiple physiological measures and independent variables will lead to greatly improved measurability of cognitive load. To demonstrate the potential of this approach, we used data from a driving simulator study in which a number of physiological measures (heart rate, heart rate variability, breathing rate, skin conductance, pupil diameter, eye blink rate, eye blink duration, EEG alpha power, and EEG theta power) were analyzed. Participants performed a cognitively loading n-back task at two levels of difficulty while driving through three different traffic scenarios, each repeated four times. Cognitive load components and other coinciding mental responses were assessed by considering response patterns of multiple physiological measures in relation to multiple independent variables. With this approach, the construct validity of cognitive load is improved, which is important for interpreting results accurately. Also, the use of multiple measures and independent variables makes the measurements (when analyzed jointly) more diagnostic-that is, better able to distinguish between different cognitive load components. This in turn improves the overall external validity. With more detailed, diagnostic, and valid measures of cognitive load, the effects of cognitive load on traffic safety can be better understood, and hence possibly mitigated.

Effects of verbal tasks on driving simulator performance

We report results from a driving simulator paradigm we developed to test the fine temporal effects of verbal tasks on simultaneous tracking performance. A total of 74 undergraduate students participated in two experiments in which they controlled a cursor using the steering wheel to track a moving target and where the dependent measure was overall deviation from target. Experiment 1 tested tracking performance during slow and fast target speeds under conditions involving either no verbal input or output, passive listening to spoken prompts via headphones, or responding to spoken prompts. Experiment 2 was similar except that participants read written prompts overlain on the simulator screen instead of listening to spoken prompts. Performance in both experiments was worse during fast speeds and worst overall during responding conditions. Most significantly, fine scale time-course analysis revealed deteriorating tracking performance as participants prepared and began speaking and steadily improving performance while speaking. Additionally, post-block survey data revealed that conversation recall was best in responding conditions, and perceived difficulty increased with task complexity. Our study is the first to track temporal changes in interference at high resolution during the first hundreds of milliseconds of verbal production and comprehension. Our results are consistent with load-based theories of multitasking performance and show that language production, and, to a lesser extent, language comprehension tap resources also used for tracking. More generally, our paradigm provides a useful tool for measuring dynamical changes in tracking performance during verbal tasks due to the rapidly changing resource requirements of language production and comprehension.

Messages beyond the phone: Processing variable message signs while attending hands-free phone calls

We examined the effects of different types of cognitive distraction coming from a hands-free phone conversation on the processing of information provided by variable message signs (VMS), on driving performance indicators, and on a physiological index of mental effort (heart rate). Participants drove a route in a driving simulator and had to respond to VMS messages under three conditions: no-distraction, visuospatial distraction (attending phone calls with questions inducing visuospatial processing), and conceptual distraction (attending phone calls with questions requiring semantic memory). Results showed more errors responding to VMS messages in the visuospatial distraction condition. In addition, both types of questions increased the intraindividual variability of response distances and the heart rate, as compared to the no-distraction condition. These results provide new evidence that talking on a hands-free phone entails costs in the processing of traffic information (in particular, text messages displayed on VMS) and it increases the driver's cognitive effort. Interestingly, the cognitive distraction had no effect on the driver's control of the vehicle speed or lateral position. Therefore, the effects of potential risk factors can critically vary among the different driving subtasks due to modulatory factors, such as the level of attentional task demands (relatively high in the processing of messages on VMS, but relatively low in controlling the speed and lateral position of the vehicle in quiet traffic conditions). In consequence, the current paper provides new evidence to discuss hands-free phone policies and highlights the importance of designing technological countermeasures to prevent drivers missing critical information displayed on VMS.

Age-Related Differences in the Cognitive, Visual, and Temporal Demands of In-Vehicle Information Systems

In-vehicle information systems (IVIS) refer to a collection of features in vehicles that allow motorists to complete tasks (often unrelated to driving) while operating the vehicle. These systems may interfere, to a greater extent, with older drivers' ability to attend to the visual and cognitive demands of the driving environment. The current study sought to examine age-related differences in the visual, cognitive and temporal demands associated with IVIS interactions. Older and younger drivers completed a set of common tasks using the IVIS of a representative sample of six different vehicles while they drove along a low-density residential street. Evaluation measures included a Detection Response Task (DRT), to assess both cognitive and visual attention, and subjective measures following each condition using the NASA Task Load Index (TLX). Two age cohorts were evaluated: younger drivers between 21 and 36 years of age, and older drivers between 55 and 75 years of age. Participants completed experimental tasks involving interactions with the IVIS to achieve a specific goal (i.e., using the touch screen to tune the radio to a station; using voice commands to find a specified navigation destination, etc.). Performance of tasks varied according to different modes of interaction available in the vehicles. Older drivers took longer to complete tasks, were slower to react to stimuli, and reported higher task demand when interacting with IVIS. Older drivers stand to benefit the most from advancements in-vehicle technology, but ironically may struggle the most to use them. The results document significant age-related costs in the potential for distraction from IVIS interactions on the road.

A Novel Classification Method for a Driver's Cognitive Stress Level by Transferring Interbeat Intervals of the ECG Signal to Pictures

In this study, a novel classification method for a driver's cognitive stress level was proposed, whereby the interbeat intervals extracted from an electrocardiogram (ECG) signal were transferred to pictures, and a convolution neural network (CNN) was used to train the pictures to classify a driver's cognitive stress level. First, we defined three levels of tasks and collected the ECG signal of the driver at different cognitive stress levels by designing and performing a driving simulation experiment. We extracted the interbeat intervals and converted them to pictures according to the number of consecutive interbeat intervals in each picture. Second, the CNN model was used to train the data set to recognize the cognitive stress levels. Classification accuracies of 100%, 91.6% and 92.8% were obtained for the training set, validation set and test set, respectively, and were compared with those the BP neural network. Last, we discussed the influence of the number of interbeat intervals in each picture on the performance of the proposed classification method. The results showed that the performance initially improved with an increase in the number of interbeat intervals. A downward trend was observed when the number exceeded 40, and when the number was 40, the model performed best with the highest accuracy (98.79%) and a relatively low relative standard deviation (0.019).

Assessing the visual and cognitive demands of in-vehicle information systems

BACKGROUND

New automobiles provide a variety of features that allow motorists to perform a plethora of secondary tasks unrelated to the primary task of driving. Despite their ubiquity, surprisingly little is known about how these complex multimodal in-vehicle information systems (IVIS) interactions impact a driver's workload.

RESULTS

The current research sought to address three interrelated questions concerning this knowledge gap: (1) Are some task types more impairing than others? (2) Are some modes of interaction more distracting than others? (3) Are IVIS interactions easier to perform in some vehicles than others? Depending on the availability of the IVIS features in each vehicle, our testing involved an assessment of up to four task types (audio entertainment, calling and dialing, text messaging, and navigation) and up to three modes of interaction (e.g., center stack, auditory vocal, and the center console). The data collected from each participant provided a measure of cognitive demand, a measure of visual/manual demand, a subjective workload measure, and a measure of the time it took to complete the different tasks. The research provides empirical evidence that the workload experienced by drivers systematically varied as a function of the different tasks, modes of interaction, and vehicles that we evaluated.

CONCLUSIONS

This objective assessment suggests that many of these IVIS features are too distracting to be enabled while the vehicle is in motion. Greater consideration should be given to what interactions should be available to the driver when the vehicle is in motion rather than to what IVIS features and functions could be available to motorists.

Drowsiness measures for commercial motor vehicle operations

Timely detection of drowsiness in Commercial Motor Vehicle (C MV) operations is necessary to reduce drowsiness-related CMV crashes. This is relevant for manual driving and, paradoxically, even more so with increasing levels of driving automation. Measures available for drowsiness detection vary in reliability, validity, usability, and effectiveness. Passively recorded physiologic measures such as electroencephalography (EEG) and a variety of ocular parameters tend to accurately identify states of considerable drowsiness, but are limited in their potential to detect lower levels of drowsiness. They also do not correlate well with measures of driver performance. Objective measures of vigilant attention performance capture drowsiness reliably, but they require active driver involvement in a performance task and are prone to confounds from distraction and (lack of) motivation. Embedded performance measures of actual driving, such as lane deviation, have been found to correlate with physiologic and vigilance performance measures, yet to what extent drowsiness levels can be derived from them reliably remains a topic of investigation. Transient effects from external circumstances and behaviors - such as task load, light exposure, physical activity, and caffeine intake - may mask a driver's underlying state of drowsiness. Also, drivers differ in the degree to which drowsiness affects their driving performance, based on trait vulnerability as well as age. This paper provides a broad overview of the current science pertinent to a range of drowsiness measures, with an emphasis on those that may be most relevant for CMV operations. There is a need for smart technologies that in a transparent manner combine different measurement modalities with mathematical representations of the neurobiological processes driving drowsiness, that account for various mediators and confounds, and that are appropriately adapted to the individual driver. The research for and development of such technologies requires a multi-disciplinary approach and significant resources, but is technically within reach.

An interview study exploring Tesla drivers' behavioural adaptation

Partially automated vehicles (PAVs) have been used in real-world environments for several years since the emergence of autonomous driving. It is important to understand the effects of partial automation systems (PAS) on the understanding of drivers and their behaviour during the first months of use. In order to adapt to new vehicle technology, drivers usually exhibit specific behaviours in this stage that are not intended by the developers, namely behavioural adaptation. The present study investigated the behavioural adaptations by early PAV adopters after short-term usage. A semi-structured interview was conducted among 20  Tesla drivers who had relatively high experience (one to five months) with Autopilot, and the interviews were synthesized to understand their behavioural adaptation, mental models, and trust during the period of use. The results showed that PAV drivers had a very positive attitude towards the PAS and drivers universally engaged in secondary tasks during automated driving. They also learned from their experiences to identify relatively safe usage conditions and they employed a safety margin to avoid exposure to excessively risky situations.

Validation of auditory detection response task method for assessing the attentional effects of cognitive load

OBJECTIVES

There are 3 standardized versions of the Detection Response Task (DRT), 2 using visual stimuli (remote DRT and head-mounted DRT) and one using tactile stimuli. In this article, we present a study that proposes and validates a type of auditory signal to be used as DRT stimulus and evaluate the proposed auditory version of this method by comparing it with the standardized visual and tactile version.

METHODS

This was a within-subject design study performed in a driving simulator with 24 participants. Each participant performed 8 2-min-long driving sessions in which they had to perform 3 different tasks: driving, answering to DRT stimuli, and performing a cognitive task (n-back task). Presence of additional cognitive load and type of DRT stimuli were defined as independent variables. DRT response times and hit rates, n-back task performance, and pupil size were observed as dependent variables.

RESULTS

Significant changes in pupil size for trials with a cognitive task compared to trials without showed that cognitive load was induced properly. Each DRT version showed a significant increase in response times and a decrease in hit rates for trials with a secondary cognitive task compared to trials without. Similar and significantly better results in differences in response times and hit rates were obtained for the auditory and tactile version compared to the visual version. There were no significant differences in performance rate between the trials without DRT stimuli compared to trials with and among the trials with different DRT stimuli modalities.

CONCLUSIONS

The results from this study show that the auditory DRT version, using the signal implementation suggested in this article, is sensitive to the effects of cognitive load on driver's attention and is significantly better than the remote visual and tactile version for auditory-vocal cognitive (n-back) secondary tasks.

Investigating the impact of static roadside advertising on drivers' situation awareness

Roadside advertising has the potential to create a crash risk for drivers as it may distract attention from driving at critical times. In an on-road instrumented vehicle study, we examined if and how static advertising billboards affect drivers' situation awareness across different driving environments. Nineteen fully licensed drivers drove an instrumented vehicle around a 38 km urban test route comprising freeway, busy urban retail and arterial road sections. The route contained a number of static billboards. Drivers provided continuous verbal protocols throughout the drive. Results indicated that the structure and content of drivers' situation awareness was not appreciably affected by the billboards in any of the road environments examined. Drivers focused their attention on the billboards when driving demand was low, such as when driving on the freeway with light to moderate traffic, in lower speed zones, or when stationary. However, when drivers were required to perform a manoeuvre or driving demands increased, drivers directed less attention to the billboards and focussed their awareness on the immediate driving task. This suggests that drivers can, at least under some conditions, effectively self-regulate their attention to billboards when required to focus on the immediate traffic or driving situation.

Talking to your car can drive you to distraction

This research examined the impact of in-vehicle information system (IVIS) interactions on the driver’s cognitive workload; 257 subjects participated in a weeklong evaluation of the IVIS interaction in one of ten different model-year 2015 automobiles. After an initial assessment of the cognitive workload associated with using the IVIS, participants took the vehicle home for 5 days and practiced using the system. At the end of the 5 days of practice, participants returned and the workload of these IVIS interactions was reassessed. The cognitive workload was found to be moderate to high, averaging 3.34 on a 5-point scale and ranged from 2.37 to 4.57. The workload was associated with the intuitiveness and complexity of the system and the time it took participants to complete the interaction. The workload experienced by older drivers was significantly greater than that experienced by younger drivers performing the same operations. Practice did not eliminate the interference from IVIS interactions. In fact, IVIS interactions that were difficult on the first day were still relatively difficult to perform after a week of practice. Finally, there were long-lasting residual costs after the IVIS interactions had terminated. The higher levels of workload should serve as a caution that these voice-based interactions can be cognitively demanding and ought not to be used indiscriminately while operating a motor vehicle.