Effects of Task-Induced Fatigue in Prolonged Conditional Automated Driving

The monitoring requests on young driver's fatigue and take-over performance in prolonged conditional automated driving

INTRODUCTION

L3 automated vehicles can perform all dynamic driving tasks unless a take-over occurs due to operational limits. This issue is potentially important for young drivers who are vulnerable road users since they have skill deficits and easily evolve into aberrant driving. However, drivers lacking active involvement may be fatigued and drowsy. Previous research indicated that performing a voluntary non-driving-related task (NDRT) could keep drivers alert, but there was no difference in take-over performance with or without NDRT. Providing a monitoring request (MR) before a possible take-over request (TOR) exhibited better take-over performance in temporary automated driving. Therefore, the study aimed to investigate the effects of MR and voluntary NDRT on young drivers' fatigue and performance.

METHOD

Twenty-five young drivers experienced 60 min automated driving on a highway with low traffic density and a TOR prompted due to a collision event. A within-subjects was designed that comprised three conditions: NONE, TOR-only, and MR + TOR. Drivers were allowed to perform a self-paced phone NDRT during automated driving.

RESULTS

The PERCLOS and blink frequency data showed that playing phones could keep drivers vigilant. The take-over performance on whether taking phone had no difference, but with MRs condition exhibited better take-over performance including the shorter reaction time and the longer TTC. Subjective evaluations also showed the advantages of MRs with more safety, trust, acceptance, and lower workload.

CONCLUSIONS

Taking MRs had a positive effect on relieving fatigue and improving take-over performance. Furthermore, MRs could potentially improve the safety and acceptance of automated driving.

PRACTICAL APPLICATIONS

The MR design can be used in the automotive industry to ensure the safest interfaces between fatigue drivers and automation systems.

Psychophysiological models of hypovigilance detection: A scoping review

Hypovigilance represents a major contributor to accidents. In operational contexts, the burden of monitoring/managing vigilance often rests on operators. Recent advances in sensing technologies allow for the development of psychophysiology-based (hypo)vigilance prediction models. Still, these models remain scarcely applied to operational situations and need better understanding. The current scoping review provides a state of knowledge regarding psychophysiological models of hypovigilance detection. Records evaluating vigilance measuring tools with gold standard comparisons and hypovigilance prediction performances were extracted from MEDLINE, PsychInfo, and Inspec. Exclusion criteria comprised aspects related to language, non-empirical papers, and sleep studies. The Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS) and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were used for bias evaluation. Twenty-one records were reviewed. They were mainly characterized by participant selection and analysis biases. Papers predominantly focused on driving and employed several common psychophysiological techniques. Yet, prediction methods and gold standards varied widely. Overall, we outline the main strategies used to assess hypovigilance, their principal limitations, and we discuss applications of these models.

The impact of non-driving related tasks on the development of driver sleepiness and takeover performances in prolonged automated driving

INTRODUCTION

Vehicle automation is thought to improve road safety since numerous accidents are caused by human error. However, the lack of active involvement and monotonous driving environments due to automation may contribute to drivers' passive fatigue and sleepiness. Previous research indicated that non-driving related tasks (NDRTs) were beneficial in maintaining drivers' arousal levels but detrimental to takeover performance.

METHOD

A 3·2 mixed design (between subjects: driving condition; within subjects: takeover orders) simulator experiment was conducted to explore the development of driver sleepiness in prolonged automated driving context and the effect of NDRTs on driver sleepiness development, and to further evaluate the impact of driver sleepiness and NDRTs on takeover performance. Sixty-three participants were randomly assigned to three driving conditions, each lasting 60 min: automated driving while performing driving environment monitoring task; visual NDRTs task; and visual NDRTs with scheduled driving environment monitoring task. Two hazardous events occurring at about the 5th and 55th min needed to be handled during the respective driving.

RESULTS

Drivers performing monitoring tasks had a faster development of driver sleepiness than drivers in the other two conditions in terms of both subjective and objective indicators. Takeover performance of drivers performing monitoring task were undermined due to driver sleepiness in terms of braking and steering reaction times, the time between saccade latency and braking or steering reaction times, and so forth. Additionally, NDRTs impaired the drivers' takeover ability in terms of saccade latency, max braking pedal input, max steering velocity, minimum time to collision, and so forth. This study shows that NDRTs with scheduled road environment monitoring task improve takeover performance during prolonged automated driving by helping to maintain driver alertness.

PRACTICAL APPLICATIONS

Findings from this work provide some technical assistance in the development of driver sleepiness monitoring systems for conditionally automated vehicles.

Do you trust me? Driver responses to automated evasive maneuvers

An increasing number of Conditionally Automated Driving (CAD) systems are being developed by major automotive manufacturers. In a CAD system, the automated system is in control of the vehicle within its operational design domain. Therefore, in CAD the vehicle is capable of tactical control of the vehicle and needs to be able to maneuver evasively by braking or steering to avoid objects. During these evasive maneuvers, the driver may attempt to take back control of the vehicle by intervening. A driver interrupting a CAD vehicle while properly performing an evasive maneuver presents a potential safety risk. To investigate this issue, 36 participants were recruited to participate in a Wizard-of-Oz research study. The participants experienced one of two evasive maneuvers of moderate intensity on a test track. The evasive maneuver required the CAD system to brake or steer to avoid the box placed in the lane of travel of the test vehicle. Drivers glanced toward the obstacle but did not intervene or prepare to intervene in response to the evasive maneuver. Importantly, the drivers who chose to intervene did so safely. These findings suggest that after experiencing a CAD vehicle for a brief period, most participants trusted the system enough to not intervene during a system-initiated evasive maneuver.

Impact of non-driving related tasks while operating automated driving systems (ADS): A systematic review

Automated Driving Systems (ADS) (SAE, 2021), promise improved safety and comfort for drivers. Current technological advances have resulted in increased automation capabilities. However, with the increase in automation capabilities, there is a shift in how drivers interact with their vehicles. Drivers can now temporarily hand over the control of the driving task to ADS under certain conditions. However, with ADS in temporary control of the vehicle, drivers may choose to engage in non-driving related tasks (NDRT). The current capabilities of ADS do not allow drivers to hand over control of the driving task indefinitely. Drivers must remain aware and be ready to take back control if necessary. There is a need to better understand drivers' performance and behaviors when driving with ADS, especially when engaged in NDRTs. This literature review, therefore, aims to understand the state of knowledge on automated vehicle systems and driver distraction. This review was conducted as per PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Studies found a significant increase in takeover times while engaging in NDRTs and driving with automation active. Studies also discuss a change in driver's visual attention, with more focus given to NDRTs as compared to the front roadway. The concerning effects of increasing reaction times and decreases in visual attention can be mitigated by using interventions and studies have had success in redirecting drivers attention and reorient them to the task of driving. The review, therefore, includes a discussion of ADS and NDRT engagement and its impact on driving behaviors such as take-over times, visual attention, trust, and workload. Implications on driver safety and performance are discussed in light of this synthesis.

Supervising the self-driving car: Situation awareness and fatigue during highly automated driving

Vehicle automation promises to reduce the demands of the driving task, making driving less fatiguing, more convenient, and safer. Nevertheless, Level 3 automated vehicles rely on a human driver to be ready to resume control, requiring the driver to reconstruct situation awareness (SA) and resume the driving task. Understanding the interaction between non-driving-related task (NDRT) use, SA, and takeover capacity is important because an effective takeover is entirely dependent on, and scaffolds from, effectively reconstructed SA. While a number of studies have looked at the behavioural impact of being 'in- or on-the-loop', fewer consider the cognitive impact, particularly the consequences for SA. The present study exposed participants to an extended simulated automated drive involving two critical takeover scenarios (early- and late-drive). We compared automated vehicle (AV) operators who were required to passively monitor the vehicle to those engaging with self-selected NDRTs. Monitoring operators demonstrated lower total- and schema-specific SA count scores following a fatiguing drive compared to those engaging with self-selected NDRTs. NDRT engagement resulted in no significant difference in SA count scores early- and late-drive. Assessment of differences in the type and sensory modality of NDRTs indicated operators make fundamentally different selections about the NDRTs they engage with in an automated driving environment compared to a manual environment. The present study provides further evidence linking SA and AV operator behaviour and underscores the need to understand the role of SA in takeover capacity. Our findings suggest that although SA declines over time regardless of driving task requirements (Monitoring versus NDRT engagement), NDRT use may facilitate better SA construction, with implications for the regulation of NDRT use in AVs as the technology progresses.

Fatigue and Secondary Media Impacts in the Automated Vehicle: A Multidimensional State Perspective

Safety researchers increasingly recognize the impacts of task-induced fatigue on vehicle driving behavior. The current study (N = 180) explored the use of a multidimensional fatigue measure, the Driver Fatigue Questionnaire (DFQ), to test the impacts of vehicle automation, secondary media use, and driver personality on fatigue states and performance in a driving simulator. Secondary media included a trivia game and a cellphone conversation. Simulated driving induced large-magnitude fatigue states in participants, including tiredness, confusion, coping through self-comforting, and muscular symptoms. Consistent with previous laboratory and field studies, dispositional fatigue proneness predicted increases in state fatigue during the drive, especially tiredness, irrespective of automation level and secondary media. Similar to previous studies, automation slowed braking response to the emergency event following takeover but did not affect fatigue. Secondary media use relieved subjective fatigue and improved lateral control but did not affect emergency braking. Confusion was, surprisingly, associated with faster braking, and tiredness was associated with impaired control of lateral position of the vehicle. These associations were not moderated by the experimental factors. Overall, data support the use of multidimensional assessments of both fatigue symptoms and information-processing components for evaluating safety impacts of interventions for fatigue.

Non-driving related tasks' effects on takeover and manual driving behavior in a real driving setting: A differentiation approach based on task switching and modality shifting

Many studies on effects of non-driving related tasks in the context of SAE Level3 automated driving have been conducted in driving simulator settings applying standardized tasks. Thereby internal validity is favored over external validity. To assess the influence of engagement in three natural non-driving related tasks on takeover behavior in the context of SAE Level3 automated driving, we conducted an experiment on a test track with a sample of naïve participants from the general public. We used a Wizard-of-Oz vehicle to simulate a SAE Level 3 traffic jam function in a real driving setting. To measure effects of compatibility between non-driving related tasks and driving task on subsequent takeover behavior and following manual driving behavior, participants played Tetris, watched a documentary film and read a text and typed a summary of it. After approx. 15 min, each non-driving related task was interrupted by a request to intervene. In the manual driving phase after the third takeover, participants encountered a balloon car positioned on their lane which they had to evade. Results show longer takeover times in the film and text condition compared to the Tetris condition. Implications on theory and practice are discussed.

Does a faster takeover necessarily mean it is better? A study on the influence of urgency and takeover-request lead time on takeover performance and safety

During conditionally automated driving, drivers are sometimes required to take over control of the vehicle if a so-called takeover request (TOR) is issued. TORs are generally issued due to system limitations. This study investigated the effect of different urgency scenarios and takeover-request lead times (TORlts) on takeover performance and safety. The experiment was conducted in a real vehicle-based driving simulator. Manual driving, 7-second TORlt and 5-second TORlt were each tested. Participants experienced three progressively urgent driving scenarios: one cut-in scenario and two obstacle-avoidance scenarios. The results indicate that the TORlt significantly affected takeover performance and safety. Within a certain range, the longer the TORlt, the safer the takeover. However, while takeover reaction time depended mainly on the length of the TORlt and was not significantly related to other factors, such as workload, greater workloads that were caused by the TORlt were associated with shorter reaction times and decreased safety. This is evidence that the reaction time should not be used as the preferred indicator to evaluate takeover performance and safety. Indicators, such as workload, minimum TTC, feature point distribution position and slope of the obstacle avoidance trajectory, can better measure and evaluate takeover performance and safety. This study can provide data support for takeover safety evaluation of conditionally automated driving.

Effects of whole-body vibration on driver drowsiness: A review

INTRODUCTION

Whole-body vibration has direct impacts on driver vigilance by increasing physical and cognitive stress on the driver, which leads to drowsiness, fatigue and road traffic accidents. Although sleep deprivation, sleep apnoea and alcohol consumption can also lead to driver drowsiness, exposure to steady vibration is the factor most readily controlled by changes to vehicle design, yet it has received comparatively less attention.

METHODS

This review investigated interrelationships between the various components of whole-body vibration and the physiological and cognitive parameters that lead to driver drowsiness, as well as the effects of vibration parameters (frequency, amplitude, waveform and duration). Vibrations transmitted to the driver body from the vehicle floor and/or seat have been considered for this review, whereas hand-arm vibration, shocks, acute or transient vibration were excluded from consideration.

RESULTS

Drowsiness is affected by interactions between the frequency, amplitude, waveform and duration of the vibration. Under optimal conditions, whole-body vibration can induce significant drowsiness within 30 min. Low frequency whole-body vibrations, particularly vibrations of 4-10 Hz, are most effective at inducing drowsiness. This review notes some limitations of current studies and suggests directions for future research.

CONCLUSIONS

This review demonstrated a strong causal link exists between whole-body vibration and driver drowsiness. Since driver drowsiness has been established to be a significant contributor to motor vehicle accidents, research is needed to identify ways to minimise the components of whole-body vibration that contribute to drowsiness, as well as devising more effective ways to counteract drowsiness.

PRACTICAL APPLICATIONS

By raising awareness of the vibrational factors that contribute to drowsiness, manufacturers will be prompted to design vehicles that reduce the influence of these factors.

Olfactory Facilitation of Takeover Performance in Highly Automated Driving

OBJECTIVE

This study aims to quantify the impact of olfactory stimulation and takeover modality on the performance of takeovers in conditionally automated driving.

BACKGROUND

Takeover requests are important for the safety of automated vehicles. The reaction time and subsequent performance of drivers in the takeover process are crucial for safety. In this study, peppermint was adopted as an auxiliary modality to the tactile and auditory design of takeover requests.

METHODS

Sixty participants took part in the experiment, which required participants to avoid a stalled vehicle after they were awoken from a state of light sleep by a takeover request. Takeover modality (tactile, auditory, and combined) was the within-subjects factor. In the between-subjects design, half of the participants received a peppermint odor stimulation when the takeover request occurred, and the other half received a placebo (air).

RESULTS

The presence of peppermint odor did not influence the reaction time, but participants did show signs of being more alert afterwards. For the moment of takeover, use of the auditory modality had a significant positive effect on reaction time compared to the tactile conditions.

CONCLUSION

Peppermint odor had a positive impact on drivers' takeover quality when engaged in nondriving-related activities such as light sleep, and the takeover request modalities were shown to be crucial for a safe and successful takeover.

APPLICATION

The results will be useful as a reference for developers of automated driving systems to design human-machine interfaces, shorten the driver's reaction time, and improve takeover quality.

Underload on the Road: Measuring Vigilance Decrements During Partially Automated Driving

Partially automated vehicle technology is increasingly common on-road. While this technology can provide safety benefits to drivers, it also introduces new concerns about driver attention. In particular, during partially automated driving (PAD), drivers are expected to stay vigilant so they can readily respond to important events in their environment. However, using partially automated vehicles on the highway places drivers in monotonous situations and requires them to do very little. This can place the driver in a state of cognitive underload in which they experience a very small amount of cognitive demand. In this situation, drivers can exhibit vigilance decrements which impact their ability to respond to on-road threats. This is of particular concern in situations when the partially automated vehicle fails to respond to a potentially critical situation and leaves all responsibility to safely navigate to the driver. This paper reviews situations that lead to vigilance decrements and characterizes the different methodologies of measuring driver vigilance during PAD, highlighting their advantages and limitations. Based on our reading of the literature, we summarize several factors future research on vigilance decrements in PAD should consider.

Effects of partially automated driving on the development of driver sleepiness

The objective of this study was to compare the development of sleepiness during manual driving versus level 2 partially automated driving, when driving on a motorway in Sweden. The hypothesis was that partially automated driving will lead to higher levels of fatigue due to underload. Eighty-nine drivers were included in the study using a 2 × 2 design with the conditions manual versus partially automated driving and daytime (full sleep) versus night-time (sleep deprived). The results showed that night-time driving led to markedly increased levels of sleepiness in terms of subjective sleepiness ratings, blink durations, PERCLOS, pupil diameter and heart rate. Partially automated driving led to slightly higher subjective sleepiness ratings, longer blink durations, decreased pupil diameter, slower heart rate, and higher EEG alpha and theta activity. However, elevated levels of sleepiness mainly arose from the night-time drives when the sleep pressure was high. During daytime, when the drivers were alert, partially automated driving had little or no detrimental effects on driver fatigue. Whether the negative effects of increased sleepiness during partially automated driving can be compensated by the positive effects of lateral and longitudinal driving support needs to be investigated in further studies.

Westdrive X LoopAR: An Open-Access Virtual Reality Project in Unity for Evaluating User Interaction Methods during Takeover Requests

With the further development of highly automated vehicles, drivers will engage in non-related tasks while being driven. Still, drivers have to take over control when requested by the car. Here, the question arises, how potentially distracted drivers get back into the control-loop quickly and safely when the car requests a takeover. To investigate effective human–machine interactions, a mobile, versatile, and cost-efficient setup is needed. Here, we describe a virtual reality toolkit for the Unity 3D game engine containing all the necessary code and assets to enable fast adaptations to various human–machine interaction experiments, including closely monitoring the subject. The presented project contains all the needed functionalities for realistic traffic behavior, cars, pedestrians, and a large, open-source, scriptable, and modular VR environment. It covers roughly 25 km², a package of 125 animated pedestrians, and numerous vehicles, including motorbikes, trucks, and cars. It also contains all the needed nature assets to make it both highly dynamic and realistic. The presented repository contains a C++ library made for LoopAR that enables force feedback for gaming steering wheels as a fully supported component. It also includes all necessary scripts for eye-tracking in the used devices. All the main functions are integrated into the graphical user interface of the Unity^® editor or are available as prefab variants to ease the use of the embedded functionalities. This project’s primary purpose is to serve as an open-access, cost-efficient toolkit that enables interested researchers to conduct realistic virtual reality research studies without costly and immobile simulators. To ensure the accessibility and usability of the mentioned toolkit, we performed a user experience report, also included in this paper.

Stress Evaluation in Simulated Autonomous and Manual Driving through the Analysis of Skin Potential Response and Electrocardiogram Signals

The evaluation of car drivers' stress condition is gaining interest as research on Autonomous Driving Systems (ADS) progresses. The analysis of the stress response can be used to assess the acceptability of ADS and to compare the driving styles of different autonomous drive algorithms. In this contribution, we present a system based on the analysis of the Electrodermal Activity Skin Potential Response (SPR) signal, aimed to reveal the driver's stress induced by different driving situations. We reduce motion artifacts by processing two SPR signals, recorded from the hands of the subjects, and outputting a single clean SPR signal. Statistical features of signal blocks are sent to a Supervised Learning Algorithm, which classifies between stress and normal driving (non-stress) conditions. We present the results obtained from an experiment using a professional driving simulator, where a group of people is asked to undergo manual and autonomous driving on a highway, facing some unexpected events meant to generate stress. The results of our experiment show that the subjects generally appear more stressed during manual driving, indicating that the autonomous drive can possibly be well received by the public. During autonomous driving, however, significant peaks of the SPR signal are evident during unexpected events. By examining the electrocardiogram signal, the average heart rate is generally higher in the manual case compared to the autonomous case. This further supports our previous findings, even if it may be due, in part, to the physical activity involved in manual driving.

Age-related differences in effects of non-driving related tasks on takeover performance in automated driving

INTRODUCTION

During SAE level 3 automated driving, the driver's role changes from active driver to fallback-ready driver. Drowsiness is one of the factors that may degrade driver's takeover performance. This study aimed to investigate effects of non-driving related tasks (NDRTs) to counter driver's drowsiness with a Level 3 system activated and to improve successive takeover performance in a critical situation. A special focus was placed on age-related differences in the effects.

METHOD

Participants of three age groups (younger, middle-aged, older) drove the Level 3 system implemented in a high-fidelity motion-based driving simulator for about 30 min under three experiment conditions: without NDRT, while watching a video clip, and while switching between watching a video clip and playing a game. The Karolinska Sleepiness Scale and eyeblink duration measured driver drowsiness. At the end of the drive, the drivers had to take over control of the vehicle and manually change the lane to avoid a collision. Reaction time and steering angle variability were measured to evaluate the two aspects of driving performance.

RESULTS

For younger drivers, both single and multiple NDRT engagements countered the development of driver drowsiness during automated driving, and their takeover performance was equivalent to or better than their performance without NDRT engagement. For older drivers, NDRT engagement did not affect the development of drowsiness but degraded takeover performance especially under the multiple NDRT engagement condition. The results for middle-aged drivers fell at an intermediate level between those for younger and older drivers. Practical Applications: The present findings do not support general recommendations of NDRT engagement to counter drowsiness during automated driving. This study is especially relevant to the automotive industry's search for options that will ensure the safest interfaces between human drivers and automation systems.