Effects of partially automated driving on the development of driver sleepiness

Development of sleepiness in professional truck drivers: Real-road testing for driver drowsiness and attention warning (DDAW) system evaluation

All new vehicle types within the European Union must now be equipped with a driver drowsiness and attention warning system starting from 2022. The specific requirements for the test procedure necessary for type approval are defined in the Annex of EU Regulation C/2021/2639. The objectives of this study were to: (i) investigate how sleepiness develops in professional truck drivers under real-road driving conditions; and (ii) assess the feasibility of a test procedure for validating driver drowsiness and attention warning systems according to the EU regulation. Twenty-four professional truck drivers participated in the test. They drove for 180 km on a dual-lane motorway, first during daytime after a normal night's sleep and then at nighttime after being awake since early morning. The results showed higher sleepiness levels during nighttime driving compared with daytime, with a faster increase in sleepiness with distance driven, especially during the night. Psychomotor vigilance task results corroborated these findings. From a driver drowsiness and attention warning testing perspective, the study design with sleep-deprived drivers at night was successful in inducing the targeted sleepiness level of a Karolinska Sleepiness Scale score of ≥ 8. Many drivers who reported a Karolinska Sleepiness Scale ≥ 8 during the drives also acknowledged feeling sleepy in the post-drive questionnaire. Reaching high levels of sleepiness on real roads during daytime is more problematic, not the least from legal and ethical perspectives as higher traffic densities during the daytime lead to increased risks.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

What is a blink? Classifying and characterizing blinks in eye openness signals

Blinks, the closing and opening of the eyelids, are used in a wide array of fields where human function and behavior are studied. In data from video-based eye trackers, blink rate and duration are often estimated from the pupil-size signal. However, blinks and their parameters can be estimated only indirectly from this signal, since it does not explicitly contain information about the eyelid position. We ask whether blinks detected from an eye openness signal that estimates the distance between the eyelids (EO blinks) are comparable to blinks detected with a traditional algorithm using the pupil-size signal (PS blinks) and how robust blink detection is when data quality is low. In terms of rate, there was an almost-perfect overlap between EO and PS blink (F1 score: 0.98) when the head was in the center of the eye tracker's tracking range where data quality was high and a high overlap (F1 score 0.94) when the head was at the edge of the tracking range where data quality was worse. When there was a difference in blink rate between EO and PS blinks, it was mainly due to data loss in the pupil-size signal. Blink durations were about 60 ms longer in EO blinks compared to PS blinks. Moreover, the dynamics of EO blinks was similar to results from previous literature. We conclude that the eye openness signal together with our proposed blink detection algorithm provides an advantageous method to detect and describe blinks in greater detail.

Driver behavior while using Level 2 vehicle automation: a hybrid naturalistic study

Vehicle automation is becoming more prevalent. Understanding how drivers use this technology and its safety implications is crucial. In a 6-8 week naturalistic study, we leveraged a hybrid naturalistic driving research design to evaluate driver behavior with Level 2 vehicle automation, incorporating unique naturalistic and experimental control conditions. Our investigation covered four main areas: automation usage, system warnings, driving demand, and driver arousal, as well as secondary task engagement. While on the interstate, drivers were advised to engage Level 2 automation whenever they deemed it safe, and they complied by using it over 70% of the time. Interestingly, the frequency of system warnings increased with prolonged use, suggesting an evolving relationship between drivers and the automation features. Our data also revealed that drivers were discerning in their use of automation, opting for manual control under high driving demand conditions. Contrary to common safety concerns, our data indicated no significant rise in driver fatigue or fidgeting when using automation, compared to a control condition. Additionally, observed patterns of engagement in secondary tasks like radio listening and text messaging challenge existing assumptions about automation leading to dangerous driver distraction. Overall, our findings provide new insights into the conditions under which drivers opt to use automation and reveal a nuanced behavioral profile that emerges when automation is in use.

The impact of non-driving related tasks on the development of driver sleepiness and takeover performances in prolonged automated driving

INTRODUCTION

Vehicle automation is thought to improve road safety since numerous accidents are caused by human error. However, the lack of active involvement and monotonous driving environments due to automation may contribute to drivers' passive fatigue and sleepiness. Previous research indicated that non-driving related tasks (NDRTs) were beneficial in maintaining drivers' arousal levels but detrimental to takeover performance.

METHOD

A 3·2 mixed design (between subjects: driving condition; within subjects: takeover orders) simulator experiment was conducted to explore the development of driver sleepiness in prolonged automated driving context and the effect of NDRTs on driver sleepiness development, and to further evaluate the impact of driver sleepiness and NDRTs on takeover performance. Sixty-three participants were randomly assigned to three driving conditions, each lasting 60 min: automated driving while performing driving environment monitoring task; visual NDRTs task; and visual NDRTs with scheduled driving environment monitoring task. Two hazardous events occurring at about the 5th and 55th min needed to be handled during the respective driving.

RESULTS

Drivers performing monitoring tasks had a faster development of driver sleepiness than drivers in the other two conditions in terms of both subjective and objective indicators. Takeover performance of drivers performing monitoring task were undermined due to driver sleepiness in terms of braking and steering reaction times, the time between saccade latency and braking or steering reaction times, and so forth. Additionally, NDRTs impaired the drivers' takeover ability in terms of saccade latency, max braking pedal input, max steering velocity, minimum time to collision, and so forth. This study shows that NDRTs with scheduled road environment monitoring task improve takeover performance during prolonged automated driving by helping to maintain driver alertness.

PRACTICAL APPLICATIONS

Findings from this work provide some technical assistance in the development of driver sleepiness monitoring systems for conditionally automated vehicles.

PERCLOS-based technologies for detecting drowsiness: current evidence and future directions

Drowsiness associated with sleep loss and circadian misalignment is a risk factor for accidents and human error. The percentage of time that the eyes are more than 80% closed (PERCLOS) is one of the most validated indices used for the passive detection of drowsiness, which is increased with sleep deprivation, after partial sleep restriction, at nighttime, and by other drowsiness manipulations during vigilance tests, simulated driving, and on-road driving. However, some cases have been reported wherein PERCLOS was not affected by drowsiness manipulations, such as in moderate drowsiness conditions, in older adults, and during aviation-related tasks. Additionally, although PERCLOS is one of the most sensitive indices for detecting drowsiness-related performance impairments during the psychomotor vigilance test or behavioral maintenance of wakefulness test, no single index is currently available as an optimal marker for detecting drowsiness during driving or other real-world situations. Based on the current published evidence, this narrative review suggests that future studies should focus on: (1) standardization to minimize differences in the definition of PERCLOS between studies; (2) extensive validation using a single device that utilizes PERCLOS-based technology; (3) development and validation of technologies that integrate PERCLOS with other behavioral and/or physiological indices, because PERCLOS alone may not be sufficiently sensitive for detecting drowsiness caused by factors other than falling asleep, such as inattention or distraction; and (4) further validation studies and field trials targeting sleep disorders and trials in real-world environments. Through such studies, PERCLOS-based technology may contribute to preventing drowsiness-related accidents and human error.

(Mis-)use of standard Autopilot and Full Self-Driving (FSD) Beta: Results from interviews with users of Tesla's FSD Beta

Tesla's Full Self-Driving Beta (FSD) program introduces technology that extends the operational design domain of standard Autopilot from highways to urban roads. This research conducted 103 in-depth semi-structured interviews with users of Tesla's FSD Beta and standard Autopilot to evaluate the impact on user behavior and perception. It was found that drivers became complacent over time with Autopilot engaged, failing to monitor the system, and engaging in safety-critical behaviors, such as hands-free driving, enabled by weights placed on the steering wheel, mind wandering, or sleeping behind the wheel. Drivers' movement of eyes, hands, and feet became more relaxed with experience with Autopilot engaged. FSD Beta required constant supervision as unfinished technology, which increased driver stress and mental and physical workload as drivers had to be constantly prepared for unsafe system behavior (doing the wrong thing at the worst time). The hands-on wheel check was not considered as being necessarily effective in driver monitoring and guaranteeing safe use. Drivers adapt to automation over time, engaging in potentially dangerous behaviors. Some behavior seems to be a knowing violation of intended use (e.g., weighting the steering wheel), and other behavior reflects a misunderstanding or lack of experience (e.g., using Autopilot on roads not designed for). As unfinished Beta technology, FSD Beta can introduce new forms of stress and can be inherently unsafe. We recommend future research to investigate to what extent these behavioral changes affect accident risk and can be alleviated through driver state monitoring and assistance.

Detecting driver fatigue using heart rate variability: A systematic review

Driver fatigue detection systems have potential to improve road safety by preventing crashes and saving lives. Conventional driver monitoring systems based on driving performance and facial features may be challenged by the application of automated driving systems. This limitation could potentially be overcome by monitoring systems based on physiological measurements. Heart rate variability (HRV) is a physiological marker of interest for detecting driver fatigue that can be measured during real life driving. This systematic review investigates the relationship between HRV measures and driver fatigue, as well as the performance of HRV based fatigue detection systems. With the applied eligibility criteria, 18 articles were identified in this review. Inconsistent results can be found within the studies that investigated differences of HRV measures between alert and fatigued drivers. For studies that developed HRV based fatigue detection systems, the detection performance showed a large variation, where the detection accuracy ranged from 44% to 100%. The inconsistency and variation of the results can be caused by differences in several key aspects in the study designs. Progress in this field is needed to determine the relationship between HRV and different fatigue causal factors and its connection to driver performance. To be deployed, HRV-based fatigue detection systems need to be thoroughly tested in real life conditions with good coverage of relevant driving scenarios and a sufficient number of participants.

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects

The driver is one of the most important factors in the safety of the transportation system. The driver's perceptual characteristics are closely related to driving behavior, while electroencephalogram (EEG) as the gold standard for evaluating human perception is non-deceptive. It is essential to study driving characteristics by analyzing the driver's brain activity pattern, effectively acquiring driver perceptual characteristics, creating a direct connection between the driver's brain and external devices, and realizing information interchange. This paper first introduces the theories related to EEG, then reviews the applications of EEG in scenarios such as fatigue driving, distracted driving, and emotional driving. The limitations of existing research have been identified and the prospect of EEG application in future brain-computer interface automotive assisted driving systems have been proposed. This review provides guidance for researchers to use EEG to improve driving safety. It also offers valuable suggestions for future research.

How drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review

Drowsiness and distraction are major factors of road crashes and responsible of>35% of road fatalities. Automated driving could solve or minimize their impact, yet it is also in itself a way to promote them. Previous literature reviews and meta-analysis regarding take-overs during automated driving primarily focused on distraction rather than drowsiness. We thus present a systematic and meta-analysis literature review focused on the effect of distraction and drowsiness on take-over performance. From an initial selection of 1896 articles from databases, we obtained by applying systematic review methodology a total of 58 articles with 42 articles dedicated to distraction and 17 articles related to drowsiness. According to our analysis, we demonstrated that distraction and drowsiness increased the take-over request reaction time (TOR-RT), which could also lead to a reduction of the quality of take-overs. In addition, this longer reaction time was even more important in the case of handheld non-driving related tasks, which is important to consider as phone use is among the most frequent tasks done during automated driving. On a more methodological aspect, we also demonstrated that take-over time budget had a significant effect on TOR-RT. However, it is difficult to estimate to what extend distraction and drowsiness could impact the take-over quality, even if several elements supported safety issues. We underpinned several limits of the current methodologies applied in the study of distraction and drowsiness such as (i) the lack of additional measures related to the take-over quality (e.g., accelerations, collision rate), (ii) the many different methodologies applied to the determination of the TOR-RT (e.g., deactivation by the steering wheel, pedals, button), (iii) the high frequency of take-over requests which can lead to habituation effects, (iv) the lack of control conditions, (v) the fact that the level of drowsiness was relatively low in most studies. We thus highlighted recommendations for a better estimation of the effect of distraction and drowsiness on take-over performance. Further studies should adopt more standardized measures of TOR-RT and additional take-over quality measures, try minimizing the number of take-over requests, and carefully consider the time budget available for the use case since it influences the TOR-RT. Regarding distraction, researchers should consider the impact of tasks requiring handholding items. Concerning drowsiness, further protocols should consider the non-linearity of drowsiness and presence of micro sleeps and favor take-over requests based on drowsiness level protocols rather than on fixed duration protocols.

Effects of Automation and Fatigue on Drivers from Various Age Groups

This study explores how drivers are affected by automation when driving in rested and fatigued conditions. Eighty-nine drivers (45 females, 44 males) aged between 20 and 85 years attended driving experiments on separate days, once in a rested and once in a fatigued condition, in a counterbalanced order. The results show an overall effect of automation to significantly reduce drivers’ workload and effort. The automation had different effects, depending on the drivers’ conditions. Differences between the manual and automated mode were larger for the perceived time pressure and effort in the fatigued condition as compared to the rested condition. Frustration was higher during manual driving when fatigued, but also higher during automated driving when rested. Subjective fatigue and the percentage of eye closure (PERCLOS) were higher in the automated mode compared to manual driving mode. PERCLOS differences between the automated and manual mode were higher in the fatigued condition than in the rested condition. There was a significant interaction effect of age and automation on drivers’ PERCLOS. These results are important for the development of driver-centered automation because they show different benefits for drivers of different ages, depending on their condition (fatigued or rested).

Let Complexity Bring Clarity: A Multidimensional Assessment of Cognitive Load Using Physiological Measures

The effects of cognitive load on driver behavior and traffic safety are unclear and in need of further investigation. Reliable measures of cognitive load for use in research and, subsequently, in the development and implementation of driver monitoring systems are therefore sought. Physiological measures are of interest since they can provide continuous recordings of driver state. Currently, however, a few issues related to their use in this context are not usually taken into consideration, despite being well-known. First, cognitive load is a multidimensional construct consisting of many mental responses (cognitive load components) to added task demand. Yet, researchers treat it as unidimensional. Second, cognitive load does not occur in isolation; rather, it is part of a complex response to task demands in a specific operational setting. Third, physiological measures typically correlate with more than one mental state, limiting the inferences that can be made from them individually. We suggest that acknowledging these issues and studying multiple mental responses using multiple physiological measures and independent variables will lead to greatly improved measurability of cognitive load. To demonstrate the potential of this approach, we used data from a driving simulator study in which a number of physiological measures (heart rate, heart rate variability, breathing rate, skin conductance, pupil diameter, eye blink rate, eye blink duration, EEG alpha power, and EEG theta power) were analyzed. Participants performed a cognitively loading n-back task at two levels of difficulty while driving through three different traffic scenarios, each repeated four times. Cognitive load components and other coinciding mental responses were assessed by considering response patterns of multiple physiological measures in relation to multiple independent variables. With this approach, the construct validity of cognitive load is improved, which is important for interpreting results accurately. Also, the use of multiple measures and independent variables makes the measurements (when analyzed jointly) more diagnostic-that is, better able to distinguish between different cognitive load components. This in turn improves the overall external validity. With more detailed, diagnostic, and valid measures of cognitive load, the effects of cognitive load on traffic safety can be better understood, and hence possibly mitigated.