Can variations in visual behavior measures be good predictors of driver sleepiness? A real driving test study

The impact of non-driving related tasks on the development of driver sleepiness and takeover performances in prolonged automated driving

INTRODUCTION

Vehicle automation is thought to improve road safety since numerous accidents are caused by human error. However, the lack of active involvement and monotonous driving environments due to automation may contribute to drivers' passive fatigue and sleepiness. Previous research indicated that non-driving related tasks (NDRTs) were beneficial in maintaining drivers' arousal levels but detrimental to takeover performance.

METHOD

A 3·2 mixed design (between subjects: driving condition; within subjects: takeover orders) simulator experiment was conducted to explore the development of driver sleepiness in prolonged automated driving context and the effect of NDRTs on driver sleepiness development, and to further evaluate the impact of driver sleepiness and NDRTs on takeover performance. Sixty-three participants were randomly assigned to three driving conditions, each lasting 60 min: automated driving while performing driving environment monitoring task; visual NDRTs task; and visual NDRTs with scheduled driving environment monitoring task. Two hazardous events occurring at about the 5th and 55th min needed to be handled during the respective driving.

RESULTS

Drivers performing monitoring tasks had a faster development of driver sleepiness than drivers in the other two conditions in terms of both subjective and objective indicators. Takeover performance of drivers performing monitoring task were undermined due to driver sleepiness in terms of braking and steering reaction times, the time between saccade latency and braking or steering reaction times, and so forth. Additionally, NDRTs impaired the drivers' takeover ability in terms of saccade latency, max braking pedal input, max steering velocity, minimum time to collision, and so forth. This study shows that NDRTs with scheduled road environment monitoring task improve takeover performance during prolonged automated driving by helping to maintain driver alertness.

PRACTICAL APPLICATIONS

Findings from this work provide some technical assistance in the development of driver sleepiness monitoring systems for conditionally automated vehicles.

An on-road examination of daytime and evening driving on rural roads: physiological, subjective, eye gaze, and driving performance outcomes

Experiencing sleepiness when driving is associated with increased crash risk. An increasing number of studies have examined on-road driver sleepiness; however, these studies typically assess the effect of sleepiness during the late night or early morning hours when sleep pressure is approaching its greatest. An on-road driving study was performed to assess how a range of physiological and sleepiness measures are impacted when driving during the daytime and evening when moderate sleepiness is experienced. In total, 27 participants (14 women and 13 men) completed a driving session in a rural town lasting approximately 60 minutes, while physiological sleepiness (heart rate variability), subjective sleepiness, eye tracking data, vehicle kinematic data and GPS speed data were recorded. Daytime driving sessions began at 12:00 or 14:00, with the evening sessions beginning at 19:30 or 20:30; only a subset of participants (n = 11) completing the evening sessions (daytime and evening order counterbalanced). The results suggest reductions in the horizontal and vertical scanning ranges occurred during the initial 40 minutes of driving for both daytime and evening sessions, but with evening sessions reductions in scanning ranges occurred across the entire driving session. Moreover, during evening driving there was an increase in physiological and subjective sleepiness levels. The results demonstrate meaningful increases in sleepiness and reductions in eye scanning when driving during both the daytime and particularly in the evening. Thus, drivers need to remain vigilant when driving during the daytime and the evening.

Using long short term memory and convolutional neural networks for driver drowsiness detection

Fatigue negatively affects the safety and performance of drivers on the road. In fact, drowsiness and fatigue are the cause of a substantial number of motor vehicle accidents. Drowsiness among the drivers can be detected using variety of modalities, including electroencephalogram (EEG), eye movement, and vehicle driving dynamics. Among these EEG is highly accurate but very intrusive and cumbersome. On the other hand, vehicle driving dynamics are very easy to acquire but accuracy is not very high. Eye movement based approach is very attractive in terms of balance between these two extremes. However, eye movement based techniques normally require an eye tracking device which consists of high speed camera with sophisticated algorithm to extract eye movement related parameters such as blinking, eye closure, saccades, fixation etc. This makes eye tracking based drowsiness detection difficult to implement as a practical system, especially on an embedded platform. In this paper, authors propose to use eye images from camera directly without the need for expensive eye-tracking system. Here, eye related movements are captured by Recurrent Neural Network (RNN) to detect the drowsiness. Long Short Term Memory (LSTM) is a class of RNN which has several advantages over vanilla RNNs. In this work an array of LSTM cells are utilized to model the eye movements. Two types of LSTMs were employed: 1-D LSTM (R-LSTM) which is used as baseline and the convolutional LSTM (C-LSTM) which facilitates using 2-D images directly. Patches of size 48 × 48 around each eye were extracted from 38 subjects, participating in a simulated driving experiment. The state of vigilance among the subjects were independently assessed by power spectral analysis of multichannel electroencephalogram (EEG) signals, recorded simultaneously, and binary labels of alert and drowsy (baseline) were generated. Results show high efficacy of the proposed system. R-LSTM based approach resulted in accuracy around 82 % and C-LSTM based approach resulted in accuracy in the range of 95%-97%. Comparison is also provided with a recently published eye-tracking based approach, showing the proposed LSTM technique outperform with a wide margin.

The relation between working conditions, aberrant driving behaviour and crash propensity among taxi drivers in China

Although the taxi industry is playing an important role in Chinese everyday life, little attention has been posed towards occupational health issues concerning the taxi drivers' working conditions, driving behaviour and road safety. A cross-sectional survey was administered to 1021 taxi drivers from 21 companies in four Chinese cities and collected information about (i) sociodemographic characteristics, (ii) working conditions, (iii) frequency of daily aberrant driving behaviour, and (iv) involvement in property-damage-only (PDO) and personal injury (PI) crashes over the past two years. A hybrid bivariate model of crash involvement was specified: (i) the hybrid part concerned a latent variable model capturing unobserved traits of the taxi drivers; (ii) the bivariate part modelled jointly both types of crashes while capturing unobserved correlation between error terms. The survey answers paint a gloomy picture in terms of workload, as taxi drivers reported averages of 9.4 working hours per day and 6.7 working days per week that amount on average to about 63.0 working hours per week. Moreover, the estimates of the hybrid bivariate model reveal that increasing levels of fatigue, reckless behaviour and aggressive behaviour are positively related to a higher propensity of crash involvement. Lastly, the heavy workload is also positively correlated with the higher propensity of crashing, not only directly as a predictor of crash involvement, but also indirectly as a covariate of fatigue and aberrant driving behaviour. The findings from this study provide insights into potential strategies for preventive education and taxi industry management to improve the working conditions and hence reduce fatigue and road risk for the taxi drivers.

Modular design of fatigue detection in naturalistic driving environments

Research in driver mental fatigue is motivated by the fact that errors made by drivers often have life-threatening consequences. This paper proposes a new modular design approach for the early detection of driver fatigue system taking into account optimisation of system performance using particle swarm optimisation (PSO). The proposed system is designed and implemented using an existing dataset that was simultaneously collected from participants and vehicles in a naturalistic environment. Four types of data are considered as fatigue-related metrics including: vehicle acceleration, vehicle rotation pattern, driver's head position and driver's head rotation. The driver's blink rate data is used in this work as a proxy for ground truth for the classification algorithm. The collected data elements are initially fed to input modules represented by ternary neural network classifiers that estimates alertness. A Bayesian algorithm with PSO is then used to combine and optimise detection performance based on the number of existing input modules as well as their output states. Performance of the developed fatigue-detection system is assessed experimentally with a small data samples of driver trips. The obtained results are found in agreement with the state-of-the-art in terms of accuracy (90.4%), sensitivity (92.6%) and specificity (90.7%). These results are achieved with significant design flexibility and robustness against partial loss of input data source(s). However, due to small sample size of dataset (N = 3), a larger dataset need to be tested with the same system framework to generalise the findings of this work.

Stationary gaze entropy predicts lane departure events in sleep-deprived drivers

Performance decrement associated with sleep deprivation is a leading contributor to traffic accidents and fatalities. While current research has focused on eye blink parameters as physiological indicators of driver drowsiness, little is understood of how gaze behaviour alters as a result of sleep deprivation. In particular, the effect of sleep deprivation on gaze entropy has not been previously examined. In this randomised, repeated measures study, 9 (4 male, 5 female) healthy participants completed two driving sessions in a fully instrumented vehicle (1 after a night of sleep deprivation and 1 after normal sleep) on a closed track, during which eye movement activity and lane departure events were recorded. Following sleep deprivation, the rate of fixations reduced while blink rate and duration as well as saccade amplitude increased. In addition, stationary and transition entropy of gaze also increased following sleep deprivation as well as with amount of time driven. An increase in stationary gaze entropy in particular was associated with higher odds of a lane departure event occurrence. These results highlight how fatigue induced by sleep deprivation and time-on-task effects can impair drivers’ visual awareness through disruption of gaze distribution and scanning patterns.