Assessment of driver drowsiness using electroencephalogram signals based on multiple functional brain networks

Augmented Recognition of Distracted Driving State Based on Electrophysiological Analysis of Brain Network

In this study, we propose an electrophysiological analysis-based brain network method for the augmented recognition of different types of distractions during driving. Driver distractions, such as cognitive processing and visual disruptions during driving, lead to distinct alterations in the electroencephalogram (EEG) signals and the extracted brain networks. We designed and conducted a simulated experiment comprising 4 distracted driving subtasks. Three connectivity indices, including both linear and nonlinear synchronization measures, were chosen to construct the brain network. By computing connectivity strengths and topological features, we explored the potential relationship between brain network configurations and states of driver distraction. Statistical analysis of network features indicates substantial differences between normal and distracted states, suggesting a reconfiguration of the brain network under distracted conditions. Different brain network features and their combinations are fed into varied machine learning classifiers to recognize the distracted driving states. The results indicate that XGBoost demonstrates superior adaptability, outperforming other classifiers across all selected network features. For individual networks, features constructed using synchronization likelihood (SL) achieved the highest accuracy in distinguishing between cognitive and visual distraction. The optimal feature set from 3 network combinations achieves an accuracy of 95.1% for binary classification and 88.3% for ternary classification of normal, cognitively distracted, and visually distracted driving states. The proposed method could accomplish the augmented recognition of distracted driving states and may serve as a valuable tool for further optimizing driver assistance systems with distraction control strategies, as well as a reference for future research on the brain-computer interface in autonomous driving.

EEG-based emotion recognition systems; comprehensive study

Emotion recognition technology through EEG signal analysis is currently a fundamental concept in artificial intelligence. This recognition has major practical implications in emotional health care, human-computer interaction, and so on. This paper provides a comprehensive study of different methods for extracting electroencephalography (EEG) features for emotion recognition from four different perspectives, including time domain features, frequency domain features, time-frequency features, and nonlinear features. We summarize the current pattern recognition methods adopted in most related works, and with the rapid development of deep learning (DL) attracting the attention of researchers in this field, we pay more attention to deep learning-based studies and analyse the characteristics, advantages, disadvantages, and applicable scenarios. Finally, the current challenges and future development directions in this field were summarized. This paper can help novice researchers in this field gain a systematic understanding of the current status of emotion recognition research based on EEG signals and provide ideas for subsequent related research.

Mental fatigue assessment by an arbitrary channel EEG based on morphological features and LSTM-CNN

In order to achieve more sensitive mental fatigue assessment (MFA) based on an arbitrary channel EEG, this study proposed a series of feature extraction methods that combine mathematical morphology (MM), as well as an LSTM-CNN architecture. Firstly, 37 subjects had their resting-state EEGs collected at rested wakefulness (RW) and after 24 h of sleep deprivation (SD) using a 30-channel EEG acquisition device, the RW and SD groups were regarded as the negative and positive groups of mental fatigue, respectively, and the EEG collection were further categorized into two conditions: eye-opened state (EO) and eye-closed state (EC). Then, since MM can reflect the morphological characteristics of EEG rhythms and their potentials relatively independently of the time-frequency analysis and phase calculation, the MM methods were found to better reflect the mental fatigue after SD statistically, whether for single features (ANOVA: p<0.000001), multiple features (clustering by K-means, t-test: p<0.01), or time series feature spaces (calculating CD, t-test: p<0.01) of a single channel. Finally, the LSTM-CNN enhanced the generalization ability when dealing with different single-channel EEG by combining GRUs with convolutional layers: comparing the AUCs of different architectures for MFA based on an arbitrary channel, LSTM-CNN (0.992) > LSTM network (0.94) > CNN (0.831) > MLP (0.754). Moreover, the use of MM also improved the accuracy of analyzed architectures, and the true/false positive rate (TPR/FPR) of the LSTM-CNN architecture for MFA based on an arbitrary channel reached 97.024 %/3.497 %, which provided a feasible solution for the arbitrary channel EEG-based MFA.

Driver drowsiness detection methods using EEG signals: a systematic review

Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.

EEG brain signals to detect the sleep health of a driver: An automated framework system based on deep learning

Mental fatigue is complex disorganization that affects the human being's efficiency in work and daily activities (e.g., driving, exercising). Encephalography is routinely used to discern this fatigue. Several automatic procedures have deployed conventional approaches to support neurologists in mental fatigue detection episodes (e.g., sleepy vs. normal). In all of the traditional procedures (e.g., support vector machine, discrimination fisher, K-nearest neighbor, and Bayesian classification), only a low accuracy is achieved when a binary classification task (e.g., tired vs. normal) is applied. The convolutional neural network model identifies the correct mathematical manipulation to turn the input into the output. In this study, a convolutional neural network is trained to recognize brain signals recorded by a wearable encephalographic cap. Unfortunately, the convolutional neural network works with large datasets. To overcome this problem, an augmentation scheme for a convolutional neural network model is essential because it can achieve higher accuracy than the traditional classifiers. The results show that our model achieved 97.3% compared to the state-of-the-art traditional methods (e.g., SVM and LDA).

Multimodal Emotion Classification Method and Analysis of Brain Functional Connectivity Networks

Since multimodal emotion classification in different human states has rarely been studied, this paper explores the emotional mechanisms of the brain functional connectivity networks after emotional stimulation. We devise a multimodal emotion classification method fusing a brain functional connectivity network based on electroencephalography (EEG) and eye gaze (ECFCEG) to study emotional mechanisms. First, the nonlinear phase lag index (PLI) and phase-locked value (PLV) are calculated to construct the multiband brain functional connectivity networks, which are then converted into binary brain networks, and the seven features of the binary brain networks are extracted. At the same time, the features of the eye gaze signals are extracted. Then, a fusion algorithm called kernel canonical correlation analysis, based on feature level and randomization (FRKCCA), is executed for feature-level fusion (FLF) of brain functional connectivity networks and eye gaze. Finally, support vector machines (SVMs) are utilized to classify positive and negative emotions in multiple frequency bands with single modal features and multimodal features. The experimental results demonstrate that multimodal complementary representation properties can effectively improve the accuracy of emotion classification, achieving a classification accuracy of 91.32±1.81%. The classification accuracy of pupil diameter in the valence dimension is higher than that of additional features. In addition, the average emotion classification effect of the valence dimension is preferable to that of arousal. Our findings demonstrate that the brain functional connectivity networks of the right brain exhibit a deficiency. In particular, the information processing ability of the right temporal (RT) and right posterior (RP) regions is weak in the low frequency after emotional stimulation; Conversely, phase synchronization of the brain functional connectivity networks based on PLI is stronger than that of PLV.

The Application of Electroencephalogram in Driving Safety: Current Status and Future Prospects

The driver is one of the most important factors in the safety of the transportation system. The driver's perceptual characteristics are closely related to driving behavior, while electroencephalogram (EEG) as the gold standard for evaluating human perception is non-deceptive. It is essential to study driving characteristics by analyzing the driver's brain activity pattern, effectively acquiring driver perceptual characteristics, creating a direct connection between the driver's brain and external devices, and realizing information interchange. This paper first introduces the theories related to EEG, then reviews the applications of EEG in scenarios such as fatigue driving, distracted driving, and emotional driving. The limitations of existing research have been identified and the prospect of EEG application in future brain-computer interface automotive assisted driving systems have been proposed. This review provides guidance for researchers to use EEG to improve driving safety. It also offers valuable suggestions for future research.

EEG Evidence Reveals Zolpidem-Related Alterations and Prognostic Value in Disorders of Consciousness

Effective treatment and accurate long-term prognostication of patients with disorders of consciousness (DOC) remain pivotal clinical issues and challenges in neuroscience. Previous studies have shown that zolpidem produces paradoxical recovery and induces similar change patterns in specific electrophysiological features in some DOC (∼6%). However, whether these specific features are neural markers of responders, and how neural features evolve over time remain unclear. Here, we capitalized on static and dynamic EEG analysis techniques to fully uncover zolpidem-induced alterations in eight patients with DOC and constructed machine-learning models to predict long-term outcomes at the single-subject level. We observed consistent patterns of change across all patients in several static features (e.g., decreased relative theta power and weakened alpha-band functional connectivity) after zolpidem administration, albeit none zolpidem responders. Based on the current evidence, previously published electrophysiological features are not neural markers for zolpidem responders. Moreover, we found that the temporal dynamics of the brain slowed down after zolpidem intake. Brain states before and after zolpidem administration could be completely characterized by the EEG features. Furthermore, long-term outcomes were accurately predicted using connectivity features. Our findings suggest that EEG neural signatures have huge potential to assess consciousness states and predict fine-grained outcomes. In summary, our results extend the understanding of the effects of zolpidem on the brain and open avenues for the application prospect of zolpidem and EEG in patients with DOC.

Physiological signal-based drowsiness detection using machine learning: Singular and hybrid signal approaches

INTRODUCTION

Drowsiness is one of the main contributors to road-related crashes and fatalities worldwide. To address this pressing global issue, researchers are continuing to develop driver drowsiness detection systems that use a variety of measures. However, most research on drowsiness detection uses approaches based on a singular metric and, as a result, fail to attain satisfactory reliability and validity to be implemented in vehicles.

METHOD

This study examines the utility of drowsiness detection based on singular and a hybrid approach. This approach considered a range of metrics from three physiological signals - electroencephalography (EEG), electrooculography (EOG), and electrocardiography (ECG) - and used subjective sleepiness indices (assessed via the Karolinska Sleepiness Scale) as ground truth. The methodology consisted of signal recording with a psychomotor vigilance test (PVT), pre-processing, extracting, and determining the important features from the physiological signals for drowsiness detection. Finally, four supervised machine learning models were developed based on the subjective sleepiness responses using the extracted physiological features to detect drowsiness levels.

RESULTS

The results illustrate that the singular physiological measures show a specific performance metric pattern, with higher sensitivity and lower specificity or vice versa. In contrast, the hybrid biosignal-based models provide a better performance profile, reducing the disparity between the two metrics.

CONCLUSIONS

The outcome of the study indicates that the selected features provided higher performance in the hybrid approaches than the singular approaches, which could be useful for future research implications. Practical Applications: Use of a hybrid approach seems warranted to improve in-vehicle driver drowsiness detection system. Practical applications will need to consider factors such as intrusiveness, ergonomics, cost-effectiveness, and user-friendliness of any driver drowsiness detection system.

Crash severity analysis of vulnerable road users using machine learning

Road crash fatality is a universal problem of the transportation system. A massive death toll caused annually due to road crash incidents, and among them, vulnerable road users (VRU) are endangered with high crash severity. This paper focuses on employing machine learning-based classification approaches for modelling injury severity of vulnerable road users-pedestrian, bicyclist, and motorcyclist. Specifically, this study aims to analyse critical features associated with different VRU groups-for pedestrian, bicyclist, motorcyclist and all VRU groups together. The critical factor of crash severity outcomes for these VRU groups is estimated in identifying the similarities and differences across different important features associated with different VRU groups. The crash data for the study is sourced from the state of Queensland in Australia for the years 2013 through 2019. The supervised machine learning algorithms considered for the empirical analysis includes the K-Nearest Neighbour (KNN), Support Vector Machine (SVM) and Random Forest (RF). In these models, 17 distinct road crash parameters are considered as input features to train models, which originate from road user characteristics, weather and environment, vehicle and driver condition, period, road characteristics and regions, traffic, and speed jurisdiction. These classification models are separately trained and tested for individual and unified VRU to assess crash severity levels. Afterwards, model performances are compared with each other to justify the best classifier where Random Forest classification models for all VRU modes are found to be comparatively robust in test accuracy: (motorcyclist: 72.30%, bicyclist: 64.45%, pedestrian: 67.23%, unified VRU: 68.57%). Based on the Random Forest model, the road crash features are ranked and compared according to their impact on crash severity classification. Furthermore, a model-based partial dependency of each road crash parameters on the severity levels is plotted and compared for each individual and unified VRU. This clarifies the tendency of road crash parameters to vary with different VRU crash severity. Based on the outcome of the comparative analysis, motorcyclists are found to be more likely exposed to higher crash severity, followed by pedestrians and bicyclists.

The effect of photic stimulation alone and in combination with sleep deprivation after a seizure-like event - reappraisal by using linear and nonlinear EEG methods

ObjectivesThe present study aimed to compare the effectiveness of different provocation tests used for the study of the 'susceptibility to seizure' by quantitative electroencephalography (EEG) analysis.MethodsEight subjects with a history of a seizure-like disturbed consciousness participated in this preliminary study. A routine EEG was carried out with photic stimulation (eyes closed and after eyes open) at the beginning of the investigation. Some days later, a sleep-deprived EEG was recorded with the same protocol. Selected epochs (in eyes closed condition) after the stimulations were analysed with Point(wise) Correlation Dimension (PD2i) and Synchronization Likelihood (SL) methods. The results were compared to those obtained by similar analysis of the resting state (control) epochs with Wilcoxon Signed Rank Test (p ≤ 0.05).ResultsIn our study, significantly lower grand mean PD2i and higher delta SL values were found in sleep-deprived state after stimulation with eyes closed compared to the control. Our results indicated a lower-dimensional, hypersynchronous state of the brain as a consequence of these combined provocations.DiscussionThis may correspond to a possible 'preictal' state of the brain. Accordingly, it is suggested that photic stimulation together with sleep deprivation seems to be more effective in provocation - especially when the stimulation was made with eyes closed.

Study of Human Tacit Knowledge Based on Electroencephalogram Signal Characteristics

Tacit knowledge is the kind of knowledge that is difficult to transfer to another person by means of writing it down or verbalizing it. In the mineral grinding process, the proficiency of the operators depends on the tacit knowledge gained from their experience and training rather than on knowledge learned from a handbook. This article proposed a method combining the electroencephalogram (EEG) signals and the industrial process to detect the proficiency of the operators in the mineral grinding process to reveal the effect of tacit knowledge on the functional cortical connection. The functional brain networks of operators were established based on partial direct coherence and directed transfer function of EEG, and the multi-classifiers were used with the graph-theoretic indexes of the FBNs as input to distinguish the trained operators (Hps) from the non-trained operators (Lps). The results showed that the brain networks of Hps had a better connectivity than those of Lps (p < 0.01), and the accuracy of classification was up to 94.2%. Our studies confirm that based on the performance of EEG features and the combination of industrial operational operation and cognitive processes, the proficiency of the operators can be detected.

Complex networks and deep learning for EEG signal analysis

Electroencephalogram (EEG) signals acquired from brain can provide an effective representation of the human's physiological and pathological states. Up to now, much work has been conducted to study and analyze the EEG signals, aiming at spying the current states or the evolution characteristics of the complex brain system. Considering the complex interactions between different structural and functional brain regions, brain network has received a lot of attention and has made great progress in brain mechanism research. In addition, characterized by autonomous, multi-layer and diversified feature extraction, deep learning has provided an effective and feasible solution for solving complex classification problems in many fields, including brain state research. Both of them show strong ability in EEG signal analysis, but the combination of these two theories to solve the difficult classification problems based on EEG signals is still in its infancy. We here review the application of these two theories in EEG signal research, mainly involving brain-computer interface, neurological disorders and cognitive analysis. Furthermore, we also develop a framework combining recurrence plots and convolutional neural network to achieve fatigue driving recognition. The results demonstrate that complex networks and deep learning can effectively implement functional complementarity for better feature extraction and classification, especially in EEG signal analysis.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue

Fatigue driving has attracted a great deal of attention for its huge influence on automobile accidents. Recognizing driving fatigue provides a primary but significant way for addressing this problem. In this paper, we first conduct the simulated driving experiments to acquire the EEG signals in alert and fatigue states. Then, for multi-channel EEG signals without pre-processing, a novel rhythm-dependent multilayer brain network (RDMB network) is developed and analyzed for driving fatigue detection. We find that there exists a significant difference between alert and fatigue states from the view of network science. Further, key sub-RDMB network based on closeness centrality are extracted. We calculate six network measures from the key sub-RDMB network and construct feature vectors to classify the alert and fatigue states. The results show that our method can respectively achieve the average accuracy of 95.28% (with sample length of 5 s), 90.25% (2 s), and 87.69% (1 s), significantly higher than compared methods. All these validate the effectiveness of RDMB network for reliable driving fatigue detection via EEG.

The short-term effects of artificially-impaired binocular vision on driving performance

Appropriate visual function is paramount to ensuring adequate driving performance and road safety. Here, we examined the influence of sudden artificially-impaired binocular vision on driving performance using a car simulator. Twenty-four young drivers (mean age 22.42 ± 3.19 years) drove under three different visual conditions (natural driving, monocular blur, and monocular occlusion) through three different traffic environments with low, medium, and high levels of complexity (highway, rural, and city, respectively). We assessed their driving performance, perceived level of task complexity, and subjectively-experienced road safety. Furthermore, as a manipulation check, we also evaluated the drivers' cardiac vagal responses, as a well-known index of task complexity. The sudden deterioration of binocular vision caused unsafe driving behaviours (distance out of the road and maximum breaking intensity) in the most complex traffic environments. Specific self-regulatory strategies (i.e. increased cardiac vagal responses) and subjective responses corroborated these results. Practitioner summary: This study provides evidence that the sudden deterioration of binocular vision has a detrimental effect on simulated driving performance. Our analysis of cardiovascular functioning shows that drivers adopt self-regulatory strategies when their binocular vision functioning is compromised. Abbreviations: VA: visual acuity; BV: binocular vision; HRV: heart rate variability; NASA: TLX: NASA-Task Load Index; SSS: Stanford Sleepiness scale; RMSSD: root mean square of successive difference; HF: high-frequency.

Sensitivity and specificity of the driver sleepiness detection methods using physiological signals: A systematic review

Driver sleepiness is a major contributor to road crashes. A system that monitors and warns the driver at a certain, critical level of arousal, could aid in reducing sleep-related crashes. To determine how driver sleepiness detection systems perform, a systematic review of the sensitivity and specificity outcomes was performed. In total, 21 studies were located that met inclusion criteria for the review. The range of sensitivity outcomes was between 39.0-98.8 % and between 73.0-98.9 % for specificity outcomes. There was considerable variation in the outcomes of the studies employing only one physiological measure (mono-signal approach), whereas, a poly-signal approach with multiple physiological signals resulted in more consistency with higher outcomes on both sensitivity and specificity metrics. Only six of the 21 studies had both sensitivity and specificity outcomes above 90.0 %, which included mono- and poly-signal approaches. Moreover, increases in the number of features used in the sleepiness detection system did not result in higher sensitivity and specificity outcomes. Overall, there was considerable variability between the studies reviewed, including measures of ground truth, the features employed and the machine learning approach of the systems. A critical need for progressing any system is a revalidation of the system on a new sample of users. These aspects indicate considerable progress is needed with physiological-based driver sleepiness systems before they are at a sufficient standard to be deployed on-road.

Detecting Unfavorable Driving States in Electroencephalography Based on a PCA Sample Entropy Feature and Multiple Classification Algorithms

Unfavorable driving states can cause a large number of vehicle crashes and are significant factors in leading to traffic accidents. Hence, the aim of this research is to design a robust system to detect unfavorable driving states based on sample entropy feature analysis and multiple classification algorithms. Multi-channel Electroencephalography (EEG) signals are recorded from 16 participants while performing two types of driving tasks. For the purpose of selecting optimal feature sets for classification, principal component analysis (PCA) is adopted for reducing dimensionality of feature sets. Multiple classification algorithms, namely, K nearest neighbor (KNN), decision tree (DT), support vector machine (SVM) and logistic regression (LR) are employed to improve the accuracy of unfavorable driving state detection. We use 10-fold cross-validation to assess the performance of the proposed systems. It is found that the proposed detection system, based on PCA features and the cubic SVM classification algorithm, shows robustness as it obtains the highest accuracy of 97.81%, sensitivity of 96.93%, specificity of 98.73% and precision of 98.75%. Experimental results show that the system we designed can effectively monitor unfavorable driving states.

Classification of Drowsiness Levels Based on a Deep Spatio-Temporal Convolutional Bidirectional LSTM Network Using Electroencephalography Signals

Non-invasive brain-computer interfaces (BCI) have been developed for recognizing human mental states with high accuracy and for decoding various types of mental conditions. In particular, accurately decoding a pilot's mental state is a critical issue as more than 70% of aviation accidents are caused by human factors, such as fatigue or drowsiness. In this study, we report the classification of not only two mental states (i.e., alert and drowsy states) but also five drowsiness levels from electroencephalogram (EEG) signals. To the best of our knowledge, this approach is the first to classify drowsiness levels in detail using only EEG signals. We acquired EEG data from ten pilots in a simulated night flight environment. For accurate detection, we proposed a deep spatio-temporal convolutional bidirectional long short-term memory network (DSTCLN) model. We evaluated the classification performance using Karolinska sleepiness scale (KSS) values for two mental states and five drowsiness levels. The grand-averaged classification accuracies were 0.87 (±0.01) and 0.69 (±0.02), respectively. Hence, we demonstrated the feasibility of classifying five drowsiness levels with high accuracy using deep learning.