Modular design of fatigue detection in naturalistic driving environments

Fatigue at the wheel: A non-visual approach to truck driver fatigue detection by multi-feature fusion

BACKGROUND

Monitoring of long-haul truck driver fatigue state has attracted considerable interest. Conventional fatigue driving detection methods based on the physiological and visual features are scarcely applicable, due to the intrusiveness, reliability, and cost-effectiveness concerns.

METHODS

We elaborately developed a fatigue driving detection method by fusion of non-visual features derived from the customized wristbands, vehicle-mounted equipment, and trip logs. To capture the spatiotemporal information within the sequential data, the bidirectional long short-term memory network with attention mechanism was proposed to determine whether the truck driver was fatigued within a fine-grained episode of one minute. The model was validated using a natural driving dataset with nine truck drivers on real-world roads in Guiyang, China during June and July 2021.

RESULTS

Our approach yielded 99.21 %, 84.44 %, 82.01 %, 99.63 %, and 83.21 % in accuracy, precision, recall, specificity, and F1-score, respectively. Compared with the mainstream visual-based methods, our approach outperformed particularly in terms of precision and recall. Photoplethysmogram stood out as the most important feature for truck driver fatigue state detection. Vehicle load, driving forward angle, cumulative driving time, midnight, and recent working hours were found to be positively associated with the probability of fatigue driving, while the galvanic skin response, vehicle acceleration, current time, and recent rest hours had a negative relationship. Specifically, truck drivers were more likely to fatigue when driving at 20-40 km/h, braking abruptly at 5-10 m/s2, with vehicle loads over 70 tons, and driving more than 100 min consecutively.

CONCLUSIONS

Our study is among the first to harness the natural driving dataset to delve into the real-life fatigue pattern of long-haul truck drivers without disruptions on routine driving tasks. The proposed method holds pragmatic prospects by providing a privacy-preserving, robust, real-time, and non-intrusive technical pathway for truck driver fatigue monitoring.

Classifying Driving Fatigue by Using EEG Signals

Fatigue driving is one of the main reasons for the occurrence of traffic accidents. Brain-computer interface, as a human-computer interaction method based on EEG signals, can communicate with the outside world and move freely through brain signals without relying on the peripheral neuromuscular system. In this paper, a simulation driving platform composed of driving simulation equipment and driving simulation software is used to simulate the real driving process. The EEG signals of the subjects are collected through simulated driving, and the EEG of five subjects is selected as the training sample, and the remaining one is the subject. As a test sample, perform feature extraction and classification experiments, select any set of normal signals and fatigue signals recorded in the driving fatigue experiment for data analysis, and then study the classification of driver fatigue levels. Experiments have proved that the PSO-H-ELM algorithm has only about 4% advantage compared with the average accuracy of the KNN algorithm and the SVM algorithm. The gap is not as big as expected, but as a new algorithm, it is applied to the detection of fatigue EEG. The two traditional algorithms are indeed more suitable. It shows that the driver fatigue level can be judged by detecting EEG, which will provide a basis for the development of on-board, real-time driving fatigue alarm devices. It will lay the foundation for traffic management departments to intervene in driving fatigue reasonably and provide a reliable basis for minimizing traffic accidents.

Global lessons learned from naturalistic driving studies to advance traffic safety and operation research: A systematic review

The state of practice of investigating traffic safety and operation is primarily based on traditional data sources, such as spot sensors, loop detectors, and historical crash data. Recently, researchers have utilized transportation data from instrumented vehicles, driving simulators, and microsimulation modeling. However, these data sources might not represent the actual driving environment at a trajectory level and might introduce bias due to their experimental control. The shortcomings of these data sources can be overcome via Naturalistic Driving Studies (NDSs) considering the fact that NDS provides detailed real-time driving data that would help investigate the safety and operational impacts of human behavior along with other factors related to weather, traffic, and roadway geometry in a naturalistic setting. With the enormous potential of the NDS data, this study leveraged the Preferred Reporting Items for Systematic Reviews and Meta-Analyzes (PRISMA) approach to shortlist the most relevant naturalistic studies out of 2304 initial studies around the world with a focus on traffic safety and operation over the past fifteen years (2005-2020). A total of 117 studies were systematically reviewed, which were grouped into seven relevant topics, including driver behavior and performance, crash/near-crash causation, driver distraction, pedestrian/bicycle safety, intersection/traffic signal related studies, detection and prediction using NDSs data, based on their frequency of appearance in the keywords of these studies. The proper deployment of Connected and Autonomous Vehicles (CAV) require an appropriate level of human behavior integration, especially at the intimal stages where both CAV and human-driven vehicles will interact and share the same roadways in a mixed traffic environment. In order to integrate the heterogeneous nature of human behavior through behavior cloning approach, real-time trajectory-level NDS data is essential. The insights from this study revealed that NDSs could be effectively leveraged to perfect the behavior cloning to facilitate rapid and safe implementation of CAV.

Applications of brain imaging methods in driving behaviour research

Applications of neuroimaging methods have substantially contributed to the scientific understanding of human factors during driving by providing a deeper insight into the neuro-cognitive aspects of driver brain. This has been achieved by conducting simulated (and occasionally, field) driving experiments while collecting driver brain signals of various types. Here, this sector of studies is comprehensively reviewed at both macro and micro scales. At the macro scale, bibliometric aspects of these studies are analysed. At the micro scale, different themes of neuroimaging driving behaviour research are identified and the findings within each theme are synthesised. The surveyed literature has reported on applications of four major brain imaging methods. These include Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG), Functional Near-Infrared Spectroscopy (fNIRS) and Magnetoencephalography (MEG), with the first two being the most common methods in this domain. While collecting driver fMRI signal has been particularly instrumental in studying neural correlates of intoxicated driving (e.g. alcohol or cannabis) or distracted driving, the EEG method has been predominantly utilised in relation to the efforts aiming at development of automatic fatigue/drowsiness detection systems, a topic to which the literature on neuro-ergonomics of driving particularly has shown a spike of interest within the last few years. The survey also reveals that topics such as driver brain activity in semi-automated settings or neural activity of drivers with brain injuries or chronic neurological conditions have by contrast been investigated to a very limited extent. Potential topics in driving behaviour research are identified that could benefit from the adoption of neuroimaging methods in future studies. In terms of practicality, while fMRI and MEG experiments have proven rather invasive and technologically challenging for adoption in driving behaviour research, EEG and fNIRS applications have been more diverse. They have even been tested beyond simulated driving settings, in field driving experiments. Advantages and limitations of each of these four neuroimaging methods in the context of driving behaviour experiments are outlined in the paper.

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Internet of things (IoT) cloud-based applications deliver advanced solutions for smart cities to decrease traffic accidents caused by driver fatigue while driving on the road. Environmental conditions or driver behavior can ultimately lead to serious roadside accidents. In recent years, the authors have developed many low-cost, computerized, driver fatigue detection systems (DFDs) to help drivers, by using multi-sensors, and mobile and cloud-based computing architecture. To promote safe driving, these are the most current emerging platforms that were introduced in the past. In this paper, we reviewed state-of-the-art approaches for predicting unsafe driving styles using three common IoT-based architectures. The novelty of this article is to show major differences among multi-sensors, smartphone-based, and cloud-based architectures in multimodal feature processing. We discussed all of the problems that machine learning techniques faced in recent years, particularly the deep learning (DL) model, to predict driver hypovigilance, especially in terms of these three IoT-based architectures. Moreover, we performed state-of-the-art comparisons by using driving simulators to incorporate multimodal features of the driver. We also mention online data sources in this article to test and train network architecture in the field of DFDs on public available multimodal datasets. These comparisons assist other authors to continue future research in this domain. To evaluate the performance, we mention the major problems in these three architectures to help researchers use the best IoT-based architecture for detecting DFDs in a real-time environment. Moreover, the important factors of Multi-Access Edge Computing (MEC) and 5th generation (5G) networks are analyzed in the context of deep learning architecture to improve the response time of DFD systems. Lastly, it is concluded that there is a research gap when it comes to implementing the DFD systems on MEC and 5G technologies by using multimodal features and DL architecture.

Construction of medical equipment-based doctor health monitoring system

The health status of doctors has been overlooked by the society and even the doctors themselves, especially those doctors who work long hours. Their attention is always on patients, so they are more likely to ignore their own health problems. Therefore, in this paper, we propose a medical equipment-based doctor health monitoring system (hereinafter referred to as Doc-care). Doc-care can be used as a private health manager for doctors, and doctors can monitor their health indicators in real time while using medical equipment to aid diagnosis and treatment. When the doctor's health status is neglected, Doc-care can protect the doctor's health; combining with the convolutional neural network method to detect and grade the doctor's health indicators, to assess the doctor's real-time health status. After referring to the doctor's past health data in the cloud server, giving appropriate advice and predictions about the doctor's health status.

The psychomotor vigilance task: Role in the diagnosis of hepatic encephalopathy and relationship with driving ability

BACKGROUND & AIMS

Hepatic encephalopathy (HE) is a syndrome of decreased vigilance and has been associated with impaired driving ability. The aim of this study was to evaluate the psychomotor vigilance task (PVT), which is used to assess both vigilance and driving ability, in a group of patients with cirrhosis and varying degrees of HE.

METHODS

A total of 145 patients (120 males, 59 ± 10 years, model for end-stage liver disease [MELD] score 13 ± 5) underwent the PVT; a subgroup of 117 completed a driving questionnaire and a subgroup of 106 underwent the psychometric hepatic encephalopathy score (PHES) and an electroencephalogram (EEG), based on which, plus a clinical evaluation, they were classed as being unimpaired (n = 51), or as having minimal (n = 35), or mild overt HE (n = 20). All patients were followed up for an average of 13 ± 5 months in relation to the occurrence of accidents and/or traffic offences, HE-related hospitalisations and death. Sixty-six healthy volunteers evenly distributed by sex, age and education served as a reference cohort for the PVT.

RESULTS

Patients showed worse PVT performance compared with healthy volunteers, and PVT indices significantly correlated with MELD, ammonia levels, PHES and the EEG results. Significant associations were observed between neuropsychiatric performance/PVT indices and licence/driving status. PVT, PHES and EEG results all predicted HE-related hospitalisations and/or death over the follow-up period; none predicted accidents or traffic offences. However, individuals with the slowest reaction times and most lapses on the PVT were often not driving despite having a licence. When patients who had stopped driving for HE-related reasons (n = 6) were modelled as having an accident or fine over the subsequent 6 and 12 months, PVT was a predictor of accidents and traffic offences, even after correction for MELD and age.

CONCLUSIONS

The PVT is worthy of further study for the purposes of both HE and driving ability assessment.

LAY SUMMARY

Hepatic encephalopathy (HE) is a complication of advanced liver disease that can manifest as excessive sleepiness. Some patients with HE have been shown to have difficulty driving. Herein, we used a test called the Psychomotor Vigilance Task (PVT), which measures sleepiness and can also be used to assess driving competence. We showed that PVT performance is fairly stable in healthy individuals. We also showed that PVT performance parallels performance in tests which are commonly used in cirrhotic patients to measure HE. We suggest that this test is helpful in quantifying HE and identifying dangerous drivers among patients with cirrhosis.