A strategically timed verbal task improves performance and neurophysiological alertness during fatiguing drives

Drowsiness Mitigation Through Driver State Monitoring Systems: A Scoping Review

OBJECTIVE

To explore the scope of available research and to identify research gaps on in-vehicle interventions for drowsiness that utilize driver monitoring systems (DMS).

BACKGROUND

DMS are gaining popularity as a countermeasure against drowsiness. However, how these systems can be best utilized to guide driver attention is unclear.

METHODS

A scoping review was conducted in adherence to PRISMA guidelines. Five electronic databases (ACM Digital Library, Scopus, IEEE Xplore, TRID, and SAE Mobilus) were systematically searched in April 2022. Original studies examining in-vehicle drowsiness interventions that use DMS in a driving context (e.g., driving simulator and driver interviews) passed the screening. Data on study details, state detection methods, and interventions were extracted.

RESULTS

Twenty studies qualified for inclusion. Majority of interventions involved warnings (n = 16) with an auditory component (n = 14). Feedback displays (n = 4) and automation takeover (n = 4) were also investigated. Multistage interventions (n = 12) first cautioned the driver, then urged them to take an action, or initiated an automation takeover. Overall, interventions had a positive impact on sleepiness levels, driving performance, and user evaluations. Whether interventions effective for one type of sleepiness (e.g., passive vs. active fatigue) will perform well for another type is unclear.

CONCLUSION

Literature mainly focused on developing sensors and improving the accuracy of DMS, but not on the driver interactions with these technologies. More intervention studies are needed in general and for investigating their long-term effects.

APPLICATION

We list gaps and limitations in the DMS literature to guide researchers and practitioners in designing and evaluating effective safety systems for drowsy driving.

Assessment of Vigilance Level during Work: Fitting a Hidden Markov Model to Heart Rate Variability

This study aimed to enhance the real-time performance and accuracy of vigilance assessment by developing a hidden Markov model (HMM). Electrocardiogram (ECG) signals were collected and processed to remove noise and baseline drift. A group of 20 volunteers participated in the study. Their heart rate variability (HRV) was measured to train parameters of the modified hidden Markov model for a vigilance assessment. The data were collected to train the model using the Baum-Welch algorithm and to obtain the state transition probability matrix A^ and the observation probability matrix B^. Finally, the data of three volunteers with different transition patterns of mental state were selected randomly and the Viterbi algorithm was used to find the optimal state, which was compared with the actual state. The constructed vigilance assessment model had a high accuracy rate, and the accuracy rate of data prediction for these three volunteers exceeded 80%. Our approach can be used in wearable products to improve their vigilance level assessment functionality or in other fields that have key positions with high concentration requirements and monotonous repetitive work.

Fatigue and Secondary Media Impacts in the Automated Vehicle: A Multidimensional State Perspective

Safety researchers increasingly recognize the impacts of task-induced fatigue on vehicle driving behavior. The current study (N = 180) explored the use of a multidimensional fatigue measure, the Driver Fatigue Questionnaire (DFQ), to test the impacts of vehicle automation, secondary media use, and driver personality on fatigue states and performance in a driving simulator. Secondary media included a trivia game and a cellphone conversation. Simulated driving induced large-magnitude fatigue states in participants, including tiredness, confusion, coping through self-comforting, and muscular symptoms. Consistent with previous laboratory and field studies, dispositional fatigue proneness predicted increases in state fatigue during the drive, especially tiredness, irrespective of automation level and secondary media. Similar to previous studies, automation slowed braking response to the emergency event following takeover but did not affect fatigue. Secondary media use relieved subjective fatigue and improved lateral control but did not affect emergency braking. Confusion was, surprisingly, associated with faster braking, and tiredness was associated with impaired control of lateral position of the vehicle. These associations were not moderated by the experimental factors. Overall, data support the use of multidimensional assessments of both fatigue symptoms and information-processing components for evaluating safety impacts of interventions for fatigue.

Mental workload accumulation effect of mobile phone distraction in L2 autopilot mode

As automated vehicles become more common, there is a need for precise measurement and definition of when and in what ways a driver can use a mobile phone in L2 autonomous driving mode, for how long it can be used, the complexity of the call content, and the accumulated mental workload. This study uses a 2 (driving mode) × 2 (call content complexity) × 6 (driving stage) three-factor mixed experimental design to investigate the effect of these factors on the driver's mental workload by measuring the driver's performance on Detection response tasks, pupil diameter, and EEG components in various brain regions in the alpha band. The results showed that drivers' mental workload levels converge between manual and automatic driving modes as the duration of driving increases, regardless of the level of complexity of the mobile phone conversation. This suggests that mobile phone conversations can also disrupt the driver's cognitive resource balance in L2 automatic driving mode, as it increases mental workload while also impairing the normal functioning of brain functions such as cognitive control, problem solving, and judgment, thereby compromising driving safety.

Exploring the effectiveness of a digital voice assistant to maintain driver alertness in partially automated vehicles

OBJECTIVE

Vehicle automation shifts the driver's role from active operator to passive observer at the potential cost of degrading their alertness. This study investigated the role of an in-vehicle voice-based assistant (VA; conversing about traffic/road environment) to counter the disengaging and fatiguing effects of automation.

METHOD

Twenty-four participants undertook two drives- with and without VA in a partially automated vehicle. Participants were subsequently categorized into high and low participation groups (based on their proportion of vocal exchanges with VA). The effectiveness of VA was assessed based on driver alertness measured using Karolinska Sleepiness Scale (KSS), eye-based sleepiness indicators and glance behavior, NASA-TLX workload rating and time to gain motor readiness in response to take-over request and performance rating made by the drivers.

RESULTS

Paired samples t-tests comparison of alertness measures across the two drives were conducted. Lower KSS rating, larger pupil diameter, higher glances (rear-mirror, roadside vehicles and signals in the drive with VA) and higher feedback ratings of VA indicated the efficiency of VA in improving driver alertness during automation. However, there was no significant difference in alertness or glance behavior between the driver groups (high and low-PR), although the time to resume steering control was significantly lower in the higher engagement group.

CONCLUSION

The study successfully demonstrated the advantages of using a voice assistant (VA) to counter these effects of passive fatigue, for example, by reducing the time to gain motor-readiness following a TOR. The findings show that despite the low engagement in spoken conversation, active listening also positively influenced driver alertness and awareness during the drive in an automated vehicle.

Coupling neuroscience and driving simulation: A systematic review of studies on crash-risk behaviors in young drivers

OBJECTIVE

Motor vehicle crashes are a leading cause of death for adolescents and young adults. The aim of this study is to examine and discuss the state-of-the-art literature which uses neuroscience methods in the context of driving simulation to study adolescent and young adult drivers.

METHODS

We conducted a systematic English-language literature search of Ovid MEDLINE (1946-2020), PsycINFO (1967-2020), PubMed, Web of Science, SCOPUS, and CINAHL using keywords and MeSH terms. Studies were excluded if participants were not within the ages of 15-25, if the driving simulator did not include a visual monitor/computer monitor/projection screen and steering wheel and foot pedals, or brain data (specifically EEG [electroencephalogram], fNIRS [functional near-infrared spectroscopy], or fMRI [functional magnetic resonance imaging]) was not collected at the same time as driving simulation data.

RESULTS

Seventy-six full text articles of the 736 studies that met inclusion criteria were included in the final review. The 76 articles used one of the following neuroscience methods: electrophysiology, functional near-infrared spectroscopy, or functional magnetic resonance imaging. In the identified studies, there were primarily two areas of investigation pursued; driving impairment and distraction in driving. Impairment studies primarily explored the areas of drowsy/fatigued driving or alcohol-impaired driving. Studies of distracted driving primarily focused on cognitive load and auditory and visual distractors.

CONCLUSIONS

Our state of the science systematic review highlights the feasibility for coupling neuroscience with driving simulation to study the neurocorrelates of driving behaviors in the context of young drivers and neuromaturation. Findings show that, to date, most research has focused on examining brain correlates and driving behaviors related to contributing factors for fatal motor vehicle crashes. However, there remains a considerable paucity of research designed to understand underlying brain mechanisms that might otherwise facilitate greater understanding of individual variability of normative and risky driving behavior within the young driving population.

Counteracting effect of verbal ratings of sleepiness on dual task interference

The aim of the present study was to demonstrate the effect of verbal ratings on arousal in the electroencephalogram (EEG) and psychomotor vigilance test (PVT) performance. Thirty participants underwent the PVT for 40 min in three experimental conditions: (1) Rating condition, in which they verbally rated subjective sleepiness with Karolinska sleepiness scale, following pure tone sound played every 20 s during PVT, (2) No-rating condition, in which they underwent PVT with the similar sound as the Rating experiment but without the verbal rating task, and (3) Control condition, in which they underwent PVT with a no-sound stimulus and without the verbal rating task. The results show that during the first half of the task epoch, alpha power density was lower in the Rating than in the No-rating condition, while performance was not different between the conditions. During the second half of the task epoch, performance was better in the Non-rating than in the Rating condition, but no difference in the alpha power density. These results suggest that performance deterioration could be masked by the arousal effect of the dual task itself. It could also explain why the PVT performance and arousal in EEG sometimes dissociate, particularly in dual task situations.

Dangerous intersections? A review of studies of fatigue and distraction in the automated vehicle

The impacts of fatigue on the vehicle driver may change with technological advancements including automation and the increasing prevalence of potentially distracting in-car systems. This article reviews the authors' simulation studies of how fatigue, automation, and distraction may intersect as threats to safety. Distinguishing between states of active and passive fatigue supports understanding of fatigue and the development of countermeasures. Active fatigue is a stress-like state driven by overload of cognitive capabilities. Passive fatigue is produced by underload and monotony, and is associated with loss of task engagement and alertness. Our studies show that automated driving reliably elicits subjective symptoms of passive fatigue and also loss of alertness that persists following manual takeover. Passive fatigue also impairs attention and automation use in operators of Remotely Piloted Vehicles (RPVs). Use of in-vehicle media has been proposed as a countermeasure to fatigue, but such media may also be distracting. Studies tested whether various forms of phone-based media interacted with automation-induced fatigue, but effects were complex and dependent on task configuration. Selection of fatigue countermeasures should be guided by an understanding of the form of fatigue confronting the operator. System design, regulation of level of automation, managing distraction, and selection of fatigue-resilient personnel are all possible interventions for passive fatigue, but careful evaluation of interventions is necessary prior to deployment.

A novel real-time driving fatigue detection system based on wireless dry EEG

Development of techniques for detection of mental fatigue has varied applications in areas where sustaining attention is of critical importance like security and transportation. The objective of this study is to develop a novel real-time driving fatigue detection methodology based on dry Electroencephalographic (EEG) signals. The study has employed two methods in the online detection of mental fatigue: power spectrum density (PSD) and sample entropy (SE). The wavelet packets transform (WPT) method was utilized to obtain the θ (4-7 Hz), α (8-12 Hz) and β (13-30 Hz) bands frequency components for calculating corresponding PSD of the selected channels. In order to improve the fatigue detection performance, the system was individually calibrated for each subject in terms of fatigue-sensitive channels selection. Two fatigue-related indexes: ( θ+α )/ β and θ / β were computed and then fused into an integrated metric to predict the degree of driving fatigue. In the case of SE extraction, the mean of SE averaged across two EEG channels ('O1h' and 'O2h') was used for fatigue detection. Ten healthy subjects participated in our study and each of them performed two sessions of simulated driving. In each session, subjects were required to drive simulated car for 90 min without any break. The results demonstrate that our proposed methods are effective for fatigue detection. The prediction of fatigue is consistent with the observation of reaction time that was recorded during simulated driving, which is considered as an objective behavioral measure.

Prevalence of Potentially Distracting Noncare Activities and Their Effects on Vigilance, Workload, and Nonroutine Events during Anesthesia Care

Background

When workload is low, anesthesia providers may perform non–patient care activities of a clinical, educational, or personal nature. Data are limited on the incidence or impact of distractions on actual care. We examined the prevalence of self-initiated nonclinical distractions and their effects on anesthesia workload, vigilance, and the occurrence of nonroutine events.

Methods

In 319 qualifying cases in an academic medical center using a Web-based electronic medical chart, a trained observer recorded video and performed behavioral task analysis. Participant workload and response to a vigilance (alarm) light were randomly measured. Postoperatively, participants were interviewed to elicit possible nonroutine events. Two anesthesiologists reviewed each event to evaluate their association with distractions.

Results

At least one self-initiated distraction was observed in 171 cases (54%), largely during maintenance. Distractions accounted for 2% of case time and lasted 2.3 s (median). The most common distraction was personal internet use. Distractions were more common in longer cases but were not affected by case type or American Society of Anesthesiologists physical status. Workload ratings were significantly lower during distraction-containing case periods and vigilance latencies were significantly longer in cases without any distractions. Three distractions were temporally associated with, but did not cause, events.

Conclusions

Both nurse anesthetists and residents performed potentially distracting tasks of a personal and/or educational nature in a majority of cases. Self-initiated distractions were rarely associated with events. This study suggests that anesthesia professionals using sound judgment can self-manage nonclinical activities. Future efforts should focus on eliminating more cognitively absorbing and less escapable distractions, as well as training in distraction management.

Fatigue in Younger and Older Drivers: Effectiveness of an Alertness-Maintaining Task

OBJECTIVE

The aim of this study was to examine the effects of an alertness-maintaining task (AMT) in older, fatigued drivers.

BACKGROUND

Fatigue during driving increases crash risk, and previous research suggests that alertness and driving in younger adults may be improved using a secondary AMT during boring, fatigue-eliciting drives. However, the potential impact of an AMT on driving has not been investigated in older drivers whose ability to complete dual tasks has been shown to decline and therefore may be negatively affected with an AMT in driving.

METHOD

Younger ( n = 29) and older drivers ( n = 39) participated in a 50-minute simulated drive designed to induce fatigue, followed by four 10-minute sessions alternating between driving with and without an AMT.

RESULTS

Younger drivers were significantly more affected by fatigue on driving performance than were older drivers but benefitted significantly from the AMT. Older drivers did not demonstrate increased driver errors with fatigue, and driving did not deteriorate significantly during participation in the AMT condition, although their speed was significantly more variable with the AMT.

CONCLUSION

Consistent with earlier research, an AMT applied during fatiguing driving is effective in improving alertness and reducing driving errors in younger drivers. Importantly, older drivers were relatively unaffected by fatigue, and use of an AMT did not detrimentally affect their driving performance.

APPLICATION

These results support the potential use of an AMT as a new automotive technology to improve fatigue and promote driver safety, though the benefits of such technology may differ between different age groups.

The relationship between cell phone use and management of driver fatigue: It's complicated

INTRODUCTION

Voice communication may enhance performance during monotonous, potentially fatiguing driving conditions (Atchley & Chan, 2011); however, it is unclear whether safety benefits of conversation are outweighed by costs. The present study tested whether personalized conversations intended to simulate hands-free cell phone conversation may counter objective and subjective fatigue effects elicited by vehicle automation.

METHOD

A passive fatigue state (Desmond & Hancock, 2001), characterized by disengagement from the task, was induced using full vehicle automation prior to drivers resuming full control over the driving simulator. A conversation was initiated shortly after reversion to manual control. During the conversation an emergency event occurred.

RESULTS

The fatigue manipulation produced greater task disengagement and slower response to the emergency event, relative to a control condition. Conversation did not mitigate passive fatigue effects; rather, it added worry about matters unrelated to the driving task. Conversation moderately improved vehicle control, as measured by SDLP, but it failed to counter fatigue-induced slowing of braking in response to an emergency event. Finally, conversation appeared to have a hidden danger in that it reduced drivers' insights into performance impairments when in a state of passive fatigue.

CONCLUSIONS

Automation induced passive fatigue, indicated by loss of task engagement; yet, simulated cell phone conversation did not counter the subjective automation-induced fatigue. Conversation also failed to counter objective loss of performance (slower braking speed) resulting from automation. Cell phone conversation in passive fatigue states may impair drivers' awareness of their performance deficits. Practical applications: Results suggest that conversation, even using a hands-free device, may not be a safe way to reduce fatigue and increase alertness during transitions from automated to manual vehicle control.

Does attention capacity moderate the effect of driver distraction in older drivers?

With age, a decline in attention capacity may occur and this may impact driving performance especially while distracted. Although the effect of distraction on driving performance of older drivers has been investigated, the moderating effect of attention capacity on driving performance during distraction has not been investigated yet. Therefore, the aim was to investigate whether attention capacity has a moderating effect on older drivers' driving performance during visual distraction (experiment 1) and cognitive distraction (experiment 2). In a fixed-based driving simulator, older drivers completed a driving task without and with visual distraction (experiment 1, N=17, mean age 78 years) or cognitive distraction (experiment 2, N=35, mean age 76 years). Several specific driving measures of varying complexity (i.e., speed, lane keeping, following distance, braking behavior, and crashes) were investigated. In addition to these objective driving measures, subjective measures of workload and driving performance were also included. In experiment 1, crash occurrence increased with visual distraction and was negatively related to attention capacity. In experiment 2, complete stops at stop signs decreased, initiation of braking at pedestrian crossings was later, and crash occurrence increased with cognitive distraction. Interestingly, for a measure of lane keeping (i.e., standard deviation of lateral lane position (SDLP)), effects of both types of distraction were moderated by attention capacity. Despite the decrease of driving performance with distraction, participants estimated their driving performance during distraction as good. These results imply that attention capacity is important for driving. Driver assessment and training programs might therefore focus on attention capacity. Nonetheless, it is crucial to eliminate driver distraction as much as possible given the deterioration of performance on several driving measures in those with low and high attention capacity.

Fatigue in the Automated Vehicle

As vehicle operation becomes increasingly automated, driver fatigue appears to be an increasingly pressing safety issue. Trivia games have been suggested as a fatigue countermeasure, but, like cell phone use, games may be distracting. The present study investigated whether secondary media devices impacted subjective responses and driver performance, during fatiguing drives. A manipulation of full and partial vehicle automation was used to induce fatigue during simulated driving. Participants were also assigned to one of three media device conditions (control, cell phone or trivia). Subjective state response, vehicle control and reaction time to a sudden event were recorded. The media devices did help minimize the loss of task engagement and elevated distress produced by vehicle automation. We also extended findings that the media devices helped improve concurrent driver performance. However, media usage was not associated with faster response time to subsequent events, suggesting that such devices may not lastingly enhance alertness.