A Review of Recent Developments in Driver Drowsiness Detection Systems

Ultrasensitive Wearable Pressure Sensors with Stress-Concentrated Tip-Array Design for Long-Term Bimodal Identification

The great challenges for existing wearable pressure sensors are the degradation of sensing performance and weak interfacial adhesion owing to the low mechanical transfer efficiency and interfacial differences at the skin-sensor interface. Here, an ultrasensitive wearable pressure sensor is reported by introducing a stress-concentrated tip-array design and self-adhesive interface for improving the detection limit. A bipyramidal microstructure with various Young's moduli is designed to improve mechanical transfer efficiency from 72.6% to 98.4%. By increasing the difference in modulus, it also mechanically amplifies the sensitivity to 8.5 V kPa-1 with a detection limit of 0.14 Pa. The self-adhesive hydrogel is developed to strengthen the sensor-skin interface, which allows stable signals for long-term and real-time monitoring. It enables generating high signal-to-noise ratios and multifeatures when wirelessly monitoring weak pulse signals and eye muscle movements. Finally, combined with a deep learning bimodal fused network, the accuracy of fatigued driving identification is significantly increased to 95.6%.

Efficient Generalized Electroencephalography-Based Drowsiness Detection Approach with Minimal Electrodes

Drowsiness is a main factor for various costly defects, even fatal accidents in areas such as construction, transportation, industry and medicine, due to the lack of monitoring vigilance in the mentioned areas. The implementation of a drowsiness detection system can greatly help to reduce the defects and accident rates by alerting individuals when they enter a drowsy state. This research proposes an electroencephalography (EEG)-based approach for detecting drowsiness. EEG signals are passed through a preprocessing chain composed of artifact removal and segmentation to ensure accurate detection followed by different feature extraction methods to extract the different features related to drowsiness. This work explores the use of various machine learning algorithms such as Support Vector Machine (SVM), the K nearest neighbor (KNN), the Naive Bayes (NB), the Decision Tree (DT), and the Multilayer Perceptron (MLP) to analyze EEG signals sourced from the DROZY database, carefully labeled into two distinct states of alertness (awake and drowsy). Segmentation into 10 s intervals ensures precise detection, while a relevant feature selection layer enhances accuracy and generalizability. The proposed approach achieves high accuracy rates of 99.84% and 96.4% for intra (subject by subject) and inter (cross-subject) modes, respectively. SVM emerges as the most effective model for drowsiness detection in the intra mode, while MLP demonstrates superior accuracy in the inter mode. This research offers a promising avenue for implementing proactive drowsiness detection systems to enhance occupational safety across various industries.

Ultra-Wide Band Radar Empowered Driver Drowsiness Detection with Convolutional Spatial Feature Engineering and Artificial Intelligence

Driving while drowsy poses significant risks, including reduced cognitive function and the potential for accidents, which can lead to severe consequences such as trauma, economic losses, injuries, or death. The use of artificial intelligence can enable effective detection of driver drowsiness, helping to prevent accidents and enhance driver performance. This research aims to address the crucial need for real-time and accurate drowsiness detection to mitigate the impact of fatigue-related accidents. Leveraging ultra-wideband radar data collected over five minutes, the dataset was segmented into one-minute chunks and transformed into grayscale images. Spatial features are retrieved from the images using a two-dimensional Convolutional Neural Network. Following that, these features were used to train and test multiple machine learning classifiers. The ensemble classifier RF-XGB-SVM, which combines Random Forest, XGBoost, and Support Vector Machine using a hard voting criterion, performed admirably with an accuracy of 96.6%. Additionally, the proposed approach was validated with a robust k-fold score of 97% and a standard deviation of 0.018, demonstrating significant results. The dataset is augmented using Generative Adversarial Networks, resulting in improved accuracies for all models. Among them, the RF-XGB-SVM model outperformed the rest with an accuracy score of 99.58%.

Cardiac indices of driver fatigue across in-lab and on-road studies

Driver fatigue is a major contributor to road accidents. Therefore, driver assistance systems (DAS) that would monitor drivers' states may contribute to road safety. Such monitoring can potentially be achieved with input from ECG indices (e.g., heart rate). We reviewed the empirical literature on responses of cardiac measures to driver fatigue and on detecting fatigue with cardiac indices and classification algorithms. We used meta-analytical methods to explore the pooled effect sizes of different cardiac indices of fatigue, their heterogeneity, and the consistency of their responses across studies. Our large pool of studies (N = 39) allowed us to stratify the results across on-road and simulator studies. We found that despite the large heterogeneity of the effect sizes between the studies, many indices had significant pooled effect sizes across the studies, and more frequently across the on-road studies. We also found that most indices showed consistent responses across both on-road and simulator studies. Regarding the detection accuracy, we found that even on-road classification could have been as accurate as 70% with only 2-min of data. However, we could only find two on-road studies that employed fatigue classification algorithms. Overall, our findings are encouraging with respect to the prospect of using cardiac measures for detecting driver fatigue. Yet, to fully explore this possibility, there is a need for additional on-road studies that would employ a similar set of cardiac indices and detection algorithms, a unified definition of fatigue, and additional levels of fatigue than the two fatigue vs alert states.

Embracing sleep-onset complexity

Sleep is crucial for many vital functions and has been extensively studied. By contrast, the sleep-onset period (SOP), often portrayed as a mere prelude to sleep, has been largely overlooked and remains poorly characterized. Recent findings, however, have reignited interest in this transitional period and have shed light on its neural mechanisms, cognitive dynamics, and clinical implications. This review synthesizes the existing knowledge about the SOP in humans. We first examine the current definition of the SOP and its limits, and consider the dynamic and complex electrophysiological changes that accompany the descent to sleep. We then describe the interplay between internal and external processing during the wake-to-sleep transition. Finally, we discuss the putative cognitive benefits of the SOP and identify novel directions to better diagnose sleep-onset disorders.

Sleepiness and the transition from wakefulness to sleep

The transition from wakefulness to sleep is a progressive process that is reflected in the gradual loss of responsiveness, an alteration of cognitive functions, and a drastic shift in brain dynamics. These changes do not occur all at once. The sleep onset period (SOP) refers here to this period of transition between wakefulness and sleep. For example, although transitions of brain activity at sleep onset can occur within seconds in a given brain region, these changes occur at different time points across the brain, resulting in a SOP that can last several minutes. Likewise, the transition to sleep impacts cognitive and behavioral levels in a graded and staged fashion. It is often accompanied and preceded by a sensation of drowsiness and the subjective feeling of a need for sleep, also associated with specific physiological and behavioral signatures. To better characterize fluctuations in vigilance and the SOP, a multidimensional approach is thus warranted. Such a multidimensional approach could mitigate important limitations in the current classification of sleep, leading ultimately to better diagnoses and treatments of individuals with sleep and/or vigilance disorders. These insights could also be translated in real-life settings to either facilitate sleep onset in individuals with sleep difficulties or, on the contrary, prevent or control inappropriate sleep onsets.

Driver Drowsiness Multi-Method Detection for Vehicles with Autonomous Driving Functions

The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands.

Effectiveness and User Perception of an In-Vehicle Voice Warning for Hypoglycemia: Development and Feasibility Trial

BACKGROUND

Hypoglycemia is a frequent and acute complication in type 1 diabetes mellitus (T1DM) and is associated with a higher risk of car mishaps. Currently, hypoglycemia can be detected and signaled through flash glucose monitoring or continuous glucose monitoring devices, which require manual and visual interaction, thereby removing the focus of attention from the driving task. Hypoglycemia causes a decrease in attention, thereby challenging the safety of using such devices behind the wheel. Here, we present an investigation of a hands-free technology-a voice warning that can potentially be delivered via an in-vehicle voice assistant.

OBJECTIVE

This study aims to investigate the feasibility of an in-vehicle voice warning for hypoglycemia, evaluating both its effectiveness and user perception.

METHODS

We designed a voice warning and evaluated it in 3 studies. In all studies, participants received a voice warning while driving. Study 0 (n=10) assessed the feasibility of using a voice warning with healthy participants driving in a simulator. Study 1 (n=18) assessed the voice warning in participants with T1DM. Study 2 (n=20) assessed the voice warning in participants with T1DM undergoing hypoglycemia while driving in a real car. We measured participants' self-reported perception of the voice warning (with a user experience scale in study 0 and with acceptance, alliance, and trust scales in studies 1 and 2) and compliance behavior (whether they stopped the car and reaction time). In addition, we assessed technology affinity and collected the participants' verbal feedback.

RESULTS

Technology affinity was similar across studies and approximately 70% of the maximal value. Perception measure of the voice warning was approximately 62% to 78% in the simulated driving and 34% to 56% in real-world driving. Perception correlated with technology affinity on specific constructs (eg, Affinity for Technology Interaction score and intention to use, optimism and performance expectancy, behavioral intention, Session Alliance Inventory score, innovativeness and hedonic motivation, and negative correlations between discomfort and behavioral intention and discomfort and competence trust; all P<.05). Compliance was 100% in all studies, whereas reaction time was higher in study 1 (mean 23, SD 5.2 seconds) than in study 0 (mean 12.6, SD 5.7 seconds) and study 2 (mean 14.6, SD 4.3 seconds). Finally, verbal feedback showed that the participants preferred the voice warning to be less verbose and interactive.

CONCLUSIONS

This is the first study to investigate the feasibility of an in-vehicle voice warning for hypoglycemia. Drivers find such an implementation useful and effective in a simulated environment, but improvements are needed in the real-world driving context. This study is a kickoff for the use of in-vehicle voice assistants for digital health interventions.

Vision Zero and Impaired Driving: Near and Longer-Term Opportunities for Preventing Death and Injuries

Vision Zero involves the use of a systems approach to eliminate fatal and serious injuries from motor vehicle crashes by accommodating basic human limitations that lead to crashes through fundamental behavioral expectations, together with sound vehicle and road design. Alcohol-related crashes account for a significant proportion of motor vehicle crash death and injury and can be addressed in a safe road transport system. We look at near-term policy and program interventions that are known to motivate drivers to make safe drinking and driving decisions, and possibilities for using technology over the longer term to address risks resulting from driver impairment that is either inadvertent or willful high-risk behavior. From the Vision Zero perspective,"normal driving" refers to a situation where traffic and road users are operating as desired and planned. A driver in this normal driving envelope operates at a safe speed, wears a seat belt, focuses on the driving task, and is not impaired. A safe system accommodates human errors, mistakes, and misjudgments in the normal driving envelope. However, it may not be capable of compensating for deliberate violations and rule-breaking. A critical role of behavioral programs and policies is to motivate safe decisions by drivers and other road users and keep them in the normal driving envelope where they can be protected from unintentional errors by a safe system. While much progress has been made in developing and implementing impaired driving policies and programs, much potential remains in the their ability to motivate drivers to meet the fundamental expectations required in a safe system. Examples of behavioral programs and policies that have strong evidence of effectiveness but are underutilized in the U.S. include conducting periodic sobriety checkpoints, lowering the blood alcohol concentration limit for driving, and mandating the use of ignition interlock devices. While the specific interventions may differ, it is likely that the same situation of incomplete implementation of behavioral programs and policies - and consequent unrealized value to a comprehensive safe system - applies to many other nations. To reach the goal of zero deaths, a comprehensive Vision Zero program needs to address the problem of deliberate risk-taking, which can include driver impairment from alcohol or other causes and extend to dangerous and reckless driving. Advanced safety technologies offer a range of opportunities for this purpose. Cars available today and in the future will have a plethora of sensors that monitor circumstances inside and around the car. These systems can identify whether a driver is in their safe driving envelope and respond with interventions that are appropriate for the severity and nature of the risk. Interventions could range from those that are not perceivable to the driver, such as putting driver assist systems into active mode, to stronger steps such as limiting or preventing vehicle operation. Zero fatalities or serious injuries in motor vehicle crashes is possible with a systems approach that accommodates human errors and mistakes that occur with the normal driving envelope and incorporates effective responses to deliberate risk-taking outside of this envelope.

Detection of Drowsiness among Drivers Using Novel Deep Convolutional Neural Network Model

Detecting drowsiness among drivers is critical for ensuring road safety and preventing accidents caused by drowsy or fatigued driving. Research on yawn detection among drivers has great significance in improving traffic safety. Although various studies have taken place where deep learning-based approaches are being proposed, there is still room for improvement to develop better and more accurate drowsiness detection systems using behavioral features such as mouth and eye movement. This study proposes a deep neural network architecture for drowsiness detection employing a convolutional neural network (CNN) for driver drowsiness detection. Experiments involve using the DLIB library to locate key facial points to calculate the mouth aspect ratio (MAR). To compensate for the small dataset, data augmentation is performed for the 'yawning' and 'no_yawning' classes. Models are trained and tested involving the original and augmented dataset to analyze the impact on model performance. Experimental results demonstrate that the proposed CNN model achieves an average accuracy of 96.69%. Performance comparison with existing state-of-the-art approaches shows better performance of the proposed model.

DDD TinyML: A TinyML-Based Driver Drowsiness Detection Model Using Deep Learning

Driver drowsiness is one of the main causes of traffic accidents today. In recent years, driver drowsiness detection has suffered from issues integrating deep learning (DL) with Internet-of-things (IoT) devices due to the limited resources of IoT devices, which pose a challenge to fulfilling DL models that demand large storage and computation. Thus, there are challenges to meeting the requirements of real-time driver drowsiness detection applications that need short latency and lightweight computation. To this end, we applied Tiny Machine Learning (TinyML) to a driver drowsiness detection case study. In this paper, we first present an overview of TinyML. After conducting some preliminary experiments, we proposed five lightweight DL models that can be deployed on a microcontroller. We applied three DL models: SqueezeNet, AlexNet, and CNN. In addition, we adopted two pretrained models (MobileNet-V2 and MobileNet-V3) to find the best model in terms of size and accuracy results. After that, we applied the optimization methods to DL models using quantization. Three quantization methods were applied: quantization-aware training (QAT), full-integer quantization (FIQ), and dynamic range quantization (DRQ). The obtained results in terms of the model size show that the CNN model achieved the smallest size of 0.05 MB using the DRQ method, followed by SqueezeNet, AlexNet MobileNet-V3, and MobileNet-V2, with 0.141 MB, 0.58 MB, 1.16 MB, and 1.55 MB, respectively. The result after applying the optimization method was 0.9964 accuracy using DRQ in the MobileNet-V2 model, which outperformed the other models, followed by the SqueezeNet and AlexNet models, with 0.9951 and 0.9924 accuracies, respectively, using DRQ.

Real-Time Machine Learning-Based Driver Drowsiness Detection Using Visual Features

Drowsiness-related car accidents continue to have a significant effect on road safety. Many of these accidents can be eliminated by alerting the drivers once they start feeling drowsy. This work presents a non-invasive system for real-time driver drowsiness detection using visual features. These features are extracted from videos obtained from a camera installed on the dashboard. The proposed system uses facial landmarks and face mesh detectors to locate the regions of interest where mouth aspect ratio, eye aspect ratio, and head pose features are extracted and fed to three different classifiers: random forest, sequential neural network, and linear support vector machine classifiers. Evaluations of the proposed system over the National Tsing Hua University driver drowsiness detection dataset showed that it can successfully detect and alarm drowsy drivers with an accuracy up to 99%.

Driver Drowsiness Detection: A Machine Learning Approach on Skin Conductance

The majority of car accidents worldwide are caused by drowsy drivers. Therefore, it is important to be able to detect when a driver is starting to feel drowsy in order to warn them before a serious accident occurs. Sometimes, drivers are not aware of their own drowsiness, but changes in their body signals can indicate that they are getting tired. Previous studies have used large and intrusive sensor systems that can be worn by the driver or placed in the vehicle to collect information about the driver's physical status from a variety of signals that are either physiological or vehicle-related. This study focuses on the use of a single wrist device that is comfortable for the driver to wear and appropriate signal processing to detect drowsiness by analyzing only the physiological skin conductance (SC) signal. To determine whether the driver is drowsy, the study tests three ensemble algorithms and finds that the Boosting algorithm is the most effective in detecting drowsiness with an accuracy of 89.4%. The results of this study show that it is possible to identify when a driver is drowsy using only signals from the skin on the wrist, and this encourages further research to develop a real-time warning system for early detection of drowsiness.

A New ECG Data Processing Approach to Developing an Accurate Driving Fatigue Detection Framework with Heart Rate Variability Analysis and Ensemble Learning

More than 1.3 million people are killed in traffic accidents annually. Road traffic accidents are mostly caused by human error. Therefore, an accurate driving fatigue detection system is required for drivers. Most driving fatigue detection studies concentrated on improving feature engineering and classification methods. We propose a novel driving fatigue detection framework concentrating on the development of the preprocessing, feature extraction, and classification stages to improve the classification accuracy of fatigue states. The proposed driving fatigue detection framework measures fatigue using a two-electrode ECG. The resampling method and heart rate variability analysis were used to extract features from the ECG data, and an ensemble learning model was utilized to classify fatigue states. To achieve the best model performance, 40 possible scenarios were applied: a combination of 5 resampling scenarios, 2 feature extraction scenarios, and 4 classification model scenarios. It was discovered that the combination of a resampling method with a window duration of 300 s and an overlap of 270 s, 54 extracted features, and AdaBoost yielded an optimum accuracy of 98.82% for the training dataset and 81.82% for the testing dataset. Furthermore, the preprocessing resampling method had the greatest impact on the model’s performance; it is a new approach presented in this study.

Design of Fatigue Driving Behavior Detection Based on Circle Hough Transform

Chronic fatigue symptoms of jobs are risk factors that may cause errors and lead to occupational accidents. For instance, occupational injuries and traffic accidents stem from overlooking long-term fatigue. According to statistics for fatigue driving, it was found that fatigue driving is one of the main causes of traffic accidents. The resulting decrease in the quality of traffic, as well as impaired traffic flow efficiency and functioning, contributes markedly to the societal costs of fatigue. This article proposes a noninvasive physical method for fatigue detection using a machine vision image algorithm. The main technology was implemented using a software framework based on optimized skin color segmentation and edge detection, as well as eye contour extraction. By integrating machine vision and an optimized Hove transform algorithm, our method mainly identifies fatigue based on the detected target's face, head gestures, mouth aspect ratio (MAR), and eye condition, and then triggers an alarm through an intelligent auxiliary device. Our evaluation results of facial image data analysis showed that with an ideal eye threshold of 0.3, PERCLOS-80 standard, MAR, and head gesture-nod frequency, the method can be used to detect fatigue data accurately and systematically, thereby fulfilling the purpose of alerting a group of high-risk drivers and preventing them from engaging in high-risk activities in an involuntary state.

System and Method for Driver Drowsiness Detection Using Behavioral and Sensor-Based Physiological Measures

The amount of road accidents caused by driver drowsiness is one of the world's major challenges. These accidents lead to numerous fatal and non-fatal injuries which impose substantial financial strain on individuals and governments every year. As a result, it is critical to prevent catastrophic accidents and reduce the financial burden on society caused by driver drowsiness. The research community has primarily focused on two approaches to identify driver drowsiness during the last decade: intrusive and non-intrusive. The intrusive approach includes physiological measures, and the non-intrusive approach includes vehicle-based and behavioral measures. In an intrusive approach, sensors are used to detect driver drowsiness by placing them on the driver's body, whereas in a non-intrusive approach, a camera is used for drowsiness detection by identifying yawning patterns, eyelid movement and head inclination. Noticeably, most research has been conducted in driver drowsiness detection methods using only single measures that failed to produce good outcomes. Furthermore, these measures were only functional in certain conditions. This paper proposes a model that combines the two approaches, non-intrusive and intrusive, to detect driver drowsiness. Behavioral measures as a non-intrusive approach and sensor-based physiological measures as an intrusive approach are combined to detect driver drowsiness. The proposed hybrid model uses AI-based Multi-Task Cascaded Convolutional Neural Networks (MTCNN) as a behavioral measure to recognize the driver's facial features, and the Galvanic Skin Response (GSR) sensor as a physiological measure to collect the skin conductance of the driver that helps to increase the overall accuracy. Furthermore, the model's efficacy has been computed in a simulated environment. The outcome shows that the proposed hybrid model is capable of identifying the transition from awake to a drowsy state in the driver in all conditions with the efficacy of 91%.

Computer vision-based approach to detect fatigue driving and face mask for edge computing device

The fatality of road accidents in this era is alarming. According to WHO, approximately 1.30 million people die each year in road accidents. Road accidents result in significant socioeconomic losses for people, their families, and the country. The integration of modern technologies into automobiles can help to reduce the number of people killed or injured in road accidents. Most of the study and police reports claim that fatigued driving is one of the deadliest factors behind many road accidents. This paper presents a complete embedded system to detect fatigue driving using deep learning, computer vision, and heart rate monitoring with Nvidia Jetson Nano developer kit, Arduino Uno, and AD8232 heart rate module. The proposed system can monitor the driver's real-time situations, then analyze the situation to detect any fatigue conditions and act accordingly. The onboard camera module constantly monitors the driver. The frames are retrieved and analyzed by the core system that uses deep learning and computer vision techniques to verify the situation with Nvidia Jetson Nano. The driver's states are identified using eye and mouth localization approaches from 68 distinct facial landmarks. Experimentally driven threshold data is employed to classify the states. The onboard heart rate module constantly measures the heart rates and detects any fluctuation in BPM related to the drowsiness. This system uses a convolutional neural network-based deep learning framework to include additional face mask detection to cope with the current pandemic situation. The heart rate module works parallelly where the other modules work in a conditional sequential manner to ensure uninterrupted detection. It will detect any sign of drowsiness in real-time and generate the alarm. The system successfully passed the initial lab tests and some actual situation experiments with 97.44% accuracy in fatigue detection and 97.90% accuracy in face mask identification. The automatic device was able to analyze different situations of drivers (different distances of driver from the camera, various lighting conditions, wearing eyeglasses, oblique projection) more precisely and generate an alarm before the accident happened.

A Deep Learning Approach to Classify Sitting and Sleep History from Raw Accelerometry Data during Simulated Driving

Prolonged sitting and inadequate sleep can impact driving performance. Therefore, objective knowledge of a driver's recent sitting and sleep history could help reduce safety risks. This study aimed to apply deep learning to raw accelerometry data collected during a simulated driving task to classify recent sitting and sleep history. Participants (n = 84, Mean ± SD age = 23.5 ± 4.8, 49% Female) completed a seven-day laboratory study. Raw accelerometry data were collected from a thigh-worn accelerometer during a 20-min simulated drive (8:10 h and 17:30 h each day). Two convolutional neural networks (CNNs; ResNet-18 and DixonNet) were trained to classify accelerometry data into four classes (sitting or breaking up sitting and 9-h or 5-h sleep). Accuracy was determined using five-fold cross-validation. ResNet-18 produced higher accuracy scores: 88.6 ± 1.3% for activity (compared to 77.2 ± 2.6% from DixonNet) and 88.6 ± 1.1% for sleep history (compared to 75.2 ± 2.6% from DixonNet). Class activation mapping revealed distinct patterns of movement and postural changes between classes. Findings demonstrate the suitability of CNNs in classifying sitting and sleep history using thigh-worn accelerometer data collected during a simulated drive. This approach has implications for the identification of drivers at risk of fatigue-related impairment.