A novel real-time driving fatigue detection system based on wireless dry EEG

Research on shared control of robots based on hybrid brain-computer interface

BACKGROUND

With the arrival of the new generation of artificial intelligence wave, new human-robot interaction technologies continue to emerge. Brain-computer interface (BCI) offers a pathway for state monitoring and interaction control between human and robot. However, the unstable mental state reduce the accuracy of human brain intent decoding, and consequently affects the precision of BCI control.

NEW METHODS

This paper proposes a hybrid BCI-based shared control (HB-SC) method for brain-controlled robot navigation. Hybrid BCI fuses electroencephalogram (EEG) and electromyography (EMG) for mental state monitoring and interactive control to output human perception and decision. The shared control based on multi-sensory fusion integrates the special obstacle information perceived by humans with the regular environmental information perceived by the robot. In this process, valid BCI commands are screened by mental state assessment and output to a layered costmap for fusion.

RESULTS

Eight subjects participated in the navigation experiment with dynamically changing mental state levels to validate the effects of a hybrid brain-computer interface through two shared control modes. The results show that the proposed HB-SC reduces collisions by 37.50 %, improves the success rate of traversing obstacles by 25.00 %, and the navigation trajectory is more consistent with expectations.

CONCLUSIONS

The HB-SC method can dynamically and intelligently adjust command output according to different brain states, helping to reduce errors made by subjects in a unstable mental state, thereby greatly enhancing the system's safety.

The architecture of functional brain network modulated by driving under train running noise exposure

A noisy environment can considerably impact drivers' attention and fatigue, endangering driving safety. Consequently, this study designed a simulated driving experimental scenario to analyse the effects of noise generated during urban rail transit train operation on drivers' functional brain networks. The experiment recruited 16 participants, and the simulated driving scenario was conducted at noise levels of 50, 60, 70, and 80 dB. Functional connectivity between all electrode pairs across various frequency bands was evaluated using the weighted phase lag index (WPLI), and a brain network based on this was constructed. Graph theoretic analysis employed network global efficiency, degree, and clustering coefficient as metrics. Significant increases in the WPLI values of theta and alpha frequency bands were observed in high noise environments (70 dB, 80 dB), as well as enhanced brain synchronisation. Furthermore, concerning the topological metrics of brain networks, it was observed that the global efficiency of brain networks in theta and alpha frequency ranges, as well as the node degree and clustering coefficients, experienced substantial growth in high noise environments (70 dB, 80 dB) as opposed to 50 dB and 60 dB. This finding indicates that high-noise environments impact the reorganisation of functional brain networks, leading to a preference for network structures with improved global efficiency. Such findings may improve our understanding of the neural mechanisms of driving under noise exposure, and thus potentially reduce road accidents to some extent.

An Identification Method for Road Hypnosis Based on Human EEG Data

The driver in road hypnosis has not only some external characteristics, but also some internal characteristics. External features have obvious manifestations and can be directly observed. Internal features do not have obvious manifestations and cannot be directly observed. They need to be measured with specific instruments. Electroencephalography (EEG), as an internal feature of drivers, is the golden parameter for drivers' life identification. EEG is of great significance for the identification of road hypnosis. An identification method for road hypnosis based on human EEG data is proposed in this paper. EEG data on drivers in road hypnosis can be collected through vehicle driving experiments and virtual driving experiments. The collected data are preprocessed with the PSD (power spectral density) method, and EEG characteristics are extracted. The neural networks EEGNet, RNN, and LSTM are used to train the road hypnosis identification model. It is shown from the results that the model based on EEGNet has the best performance in terms of identification for road hypnosis, with an accuracy of 93.01%. The effectiveness and accuracy of the identification for road hypnosis are improved in this study. The essential characteristics for road hypnosis are also revealed. This is of great significance for improving the safety level of intelligent vehicles and reducing the number of traffic accidents caused by road hypnosis.

Driving fatigue detection based on brain source activity and ARMA model

Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.

Modulating Driver Alertness via Ambient Olfactory Stimulation: A Wearable Electroencephalography Study

Poor alertness levels and related changes in cognitive efficiency are common when performing monotonous tasks such as extended driving. Recent studies have investigated driver alertness decrement and possible strategies for modulating alertness with the goal of improving reaction times to safety critical events. However, most studies rely on subjective measures in assessing alertness changes, while the use of olfactory stimuli, which are known to be strong modulators of cognitive states, has not been commensurately explored in driving alertness settings. To address this gap, in the present study we investigated the effectiveness of olfactory stimuli in modulating the alertness state of drivers and explored the utility of electroencephalography (EEG) in developing objective brain-based tools for assessing the resulting changes in cortical activity. Olfactory stimulation induced a significant differential effect on braking reaction time. The corresponding effect to the cortical activity was characterized using EEG-derived metrics and the devised machine learning framework yielded a high discriminating accuracy (92.1%). Furthermore, neural activity in the alpha frequency band was found to be significantly associated with the observed drivers' behavioral changes. Overall, our results demonstrate the potential of olfactory stimuli to modulate the alertness state and the efficiency of EEG in objectively assessing the resulting cognitive changes.

Resting-state EEG recorded with gel-based vs. consumer dry electrodes: spectral characteristics and across-device correlations

Introduction

Recordings of electroencephalographic (EEG) rhythms and their analyses have been instrumental in basic neuroscience, clinical diagnostics, and the field of brain-computer interfaces (BCIs). While in the past such measurements have been conducted mostly in laboratory settings, recent advancements in dry electrode technology pave way to a broader range of consumer and medical application because of their greater convenience compared to gel-based electrodes.

Methods

Here we conducted resting-state EEG recordings in two groups of healthy participants using three dry-electrode devices, the PSBD Headband, the PSBD Headphones and the Muse Headband, and one standard gel electrode-based system, the NVX. We examined signal quality for various spatial and spectral ranges which are essential for cognitive monitoring and consumer applications.

Results

Distinctive characteristics of signal quality were found, with the PSBD Headband showing sensitivity in low-frequency ranges and replicating the modulations of delta, theta and alpha power corresponding to the eyes-open and eyes-closed conditions, and the NVX system performing well in capturing high-frequency oscillations. The PSBD Headphones were more prone to low-frequency artifacts compared to the PSBD Headband, yet recorded modulations in the alpha power and had a strong alignment with the NVX at the higher EEG frequencies. The Muse Headband had several limitations in signal quality.

Discussion

We suggest that while dry-electrode technology appears to be appropriate for the EEG rhythm-based applications, the potential benefits of these technologies in terms of ease of use and accessibility should be carefully weighed against the capacity of each given system.

In-Car Environment Control Using an SSVEP-Based Brain-Computer Interface with Visual Stimuli Presented on Head-Up Display: Performance Comparison with a Button-Press Interface

Controlling the in-car environment, including temperature and ventilation, is necessary for a comfortable driving experience. However, it often distracts the driver's attention, potentially causing critical car accidents. In the present study, we implemented an in-car environment control system utilizing a brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP). In the experiment, four visual stimuli were displayed on a laboratory-made head-up display (HUD). This allowed the participants to control the in-car environment by simply staring at a target visual stimulus, i.e., without pressing a button or averting their eyes from the front. The driving performances in two realistic driving tests-obstacle avoidance and car-following tests-were then compared between the manual control condition and SSVEP-BCI control condition using a driving simulator. In the obstacle avoidance driving test, where participants needed to stop the car when obstacles suddenly appeared, the participants showed significantly shorter response time (1.42 ± 0.26 s) in the SSVEP-BCI control condition than in the manual control condition (1.79 ± 0.27 s). No-response rate, defined as the ratio of obstacles that the participants did not react to, was also significantly lower in the SSVEP-BCI control condition (4.6 ± 14.7%) than in the manual control condition (20.5 ± 25.2%). In the car-following driving test, where the participants were instructed to follow a preceding car that runs at a sinusoidally changing speed, the participants showed significantly lower speed difference with the preceding car in the SSVEP-BCI control condition (15.65 ± 7.04 km/h) than in the manual control condition (19.54 ± 11.51 km/h). The in-car environment control system using SSVEP-based BCI showed a possibility that might contribute to safer driving by keeping the driver's focus on the front and thereby enhancing the overall driving performance.

Contactless Heart and Respiration Rates Estimation and Classification of Driver Physiological States Using CW Radar and Temporal Neural Networks

The measurement and analysis of vital signs are a subject of significant research interest, particularly for monitoring the driver's physiological state, which is of crucial importance for road safety. Various approaches have been proposed using contact techniques to measure vital signs. However, all of these methods are invasive and cumbersome for the driver. This paper proposes using a non-contact sensor based on continuous wave (CW) radar at 24 GHz to measure vital signs. We associate these measurements with distinct temporal neural networks to analyze the signals to detect and extract heart and respiration rates as well as classify the physiological state of the driver. This approach offers robust performance in estimating the exact values of heart and respiration rates and in classifying the driver's physiological state. It is non-invasive and requires no physical contact with the driver, making it particularly practical and safe. The results presented in this paper, derived from the use of a 1D Convolutional Neural Network (1D-CNN), a Temporal Convolutional Network (TCN), a Recurrent Neural Network particularly the Bidirectional Long Short-Term Memory (Bi-LSTM), and a Convolutional Recurrent Neural Network (CRNN). Among these, the CRNN emerged as the most effective Deep Learning approach for vital signal analysis.

Driver drowsiness detection methods using EEG signals: a systematic review

Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.

Ratio Indexes Based on Spectral Electroencephalographic Brainwaves for Assessment of Mental Involvement: A Systematic Review

BACKGROUND

This review systematically examined the scientific literature about electroencephalogram-derived ratio indexes used to assess human mental involvement, in order to deduce what they are, how they are defined and used, and what their best fields of application are. (2) Methods: The review was carried out according to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines. (3) Results: From the search query, 82 documents resulted. The majority (82%) were classified as related to mental strain, while 12% were classified as related to sensory and emotion aspects, and 6% to movement. The electroencephalographic electrode montage used was low-density in 13%, high-density in 6% and very-low-density in 81% of documents. The most used electrode positions for computation of involvement indexes were in the frontal and prefrontal cortex. Overall, 37 different formulations of involvement indexes were found. None of them could be directly related to a specific field of application. (4) Conclusions: Standardization in the definition of these indexes is missing, both in the considered frequency bands and in the exploited electrodes. Future research may focus on the development of indexes with a unique definition to monitor and characterize mental involvement.

Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities

Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.

Utilizing EEG and fNIRS for the detection of sleep-deprivation-induced fatigue and its inhibition using colored light stimulation

Drowsy driving is a common, but underestimated phenomenon in terms of associated risks as it often results in crashes causing fatalities and serious injuries. It is a challenging task to alert or reduce the driver's drowsy state using non-invasive techniques. In this study, a drowsiness reduction strategy has been developed and analyzed using exposure to different light colors and recording the corresponding electrical and biological brain activities. 31 subjects were examined by dividing them into 2 classes, a control group, and a healthy group. Fourteen EEG and 42 fNIRS channels were used to gather neurological data from two brain regions (prefrontal and visual cortices). Experiments shining 3 different colored lights have been carried out on them at certain times when there is a high probability to get drowsy. The results of this study show that there is a significant increase in HbO of a sleep-deprived participant when he is exposed to blue light. Similarly, the beta band of EEG also showed an increased response. However, the study found that there is no considerable increase in HbO and beta band power in the case of red and green light exposures. In addition to that, values of other physiological signals acquired such as heart rate, eye blinking, and self-reported Karolinska Sleepiness Scale scores validated the findings predicted by the electrical and biological signals. The statistical significance of the signals achieved has been tested using repeated measures ANOVA and t-tests. Correlation scores were also calculated to find the association between the changes in the data signals with the corresponding changes in the alertness level.

An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN

OBJECTIVES

Electroencephalogram (EEG) is often used to detect mental fatigue because of its real-time characteristic and objective nature. However, because of the individual variability of EEG among different individuals, tedious and time-consuming calibration sessions are needed.

METHODS

Therefore, we propose a multi-source domain adaptation network for inter-subject mental fatigue detection named FLDANN, which is short for focal loss based domain-adversarial training of neural network. As for mental state feature extraction, power spectrum density is extracted based on the Welch method from four sub-bands of EEG signals. The features of the source domain and target domain are fed into the FLDANN network. The contributions of FLDANN include: (1) It uses the idea of adversarial to reduce feature differences between the source and target domain. (2) A loss function named focal loss is used to assign weights to source and target domain samples.

RESULTS

The experiment result shows that when the number of the source domains increases, the classification accuracy of domain-adversarial training of neural network (DANN) gradually decreases and finally tends to be stable. The proposed method achieves an accuracy of 84.10% ± 8.75% on the SEED-VIG dataset and 65.42% ± 7.47% on the self-designed dataset. In addition, the proposed method is compared with other domain adaptation methods and the results show that the proposed method outperforms those state-of-the-art methods.

CONCLUSIONS

The result proves that the proposed method is able to solve the problem of individual differences across subjects and to solve the problem of low classification performance of multi-source domain transfer learning.

BCI system with lower-limb robot improves rehabilitation in spinal cord injury patients through short-term training: a pilot study

In the recent years, the increasing applications of brain-computer interface (BCI) in rehabilitation programs have enhanced the chances of functional recovery for patients with neurological disorders. We presented and validated a BCI system with a lower-limb robot for short-term training of patients with spinal cord injury (SCI). The cores of this system included: (1) electroencephalogram (EEG) features related to motor intention reported through experiments and used to drive the robot; (2) a decision tree to determine the training mode provided for patients with different degrees of injuries. Seven SCI patients (one American Spinal Injury Association Impairment Scale (AIS) A, three AIS B, and three AIS C) participated in the short-term training with this system. All patients could learn to use the system rapidly and maintained a high intensity during the training program. The strength of the lower limb key muscles of the patients was improved. Four AIS A/B patients were elevated to AIS C. The cumulative results indicate that clinical application of the BCI system with lower-limb robot is feasible and safe, and has potentially positive effects on SCI patients.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11571-022-09801-6.

On Fatigue Detection for Air Traffic Controllers Based on Fuzzy Fusion of Multiple Features

Fatigue detection for air traffic controllers is an important yet challenging problem in aviation safety research. Most of the existing methods for this problem are based on facial features. In this paper, we propose an ensemble learning model that combines both facial features and voice features and design a fatigue detection method through multifeature fusion, referred to as Facial and Voice Stacking (FV-Stacking). Specifically, for facial features, we first use OpenCV and Dlib libraries to extract mouth and eye areas and then employ a combination of M-Convolutional Neural Network (M-CNN) and E-Convolutional Neural Network (E-CNN) to determine the state of mouth and eye closure based on five features, i.e., blinking times, average blinking time, average blinking interval, Percentage of Eyelid Closure over the Pupil over Time (PERCLOS), and Frequency of Open Mouth (FOM). For voice features, we extract the Mel-Frequency Cepstral Coefficients (MFCC) features of speech. Such facial features and voice features are fused through a carefully designed stacking model for fatigue detection. Real-life experiments are conducted on 14 air traffic controllers in Southwest Air Traffic Management Bureau of Civil Aviation of China. The results show that the proposed FV-Stacking method achieves a detection accuracy of 97%, while the best accuracy achieved by a single model is 92% and the best accuracy achieved by the state-of-the-art detection methods is 88%.

EEG-based brain functional connectivity representation using amplitude locking value for fatigue-driving recognition

It has been shown that brain functional networks constructed from electroencephalographic signals (EEG) continuously change topology as brain fatigue increases, and extracting the topological properties of the network can characterize the degree of brain fatigue. However, the traditional brain function network construction process often selects only the amplitude or phase components of the signal to measure the relationship between brain regions, and the use of a single component of the signal to construct a brain function network for analysis is rather one-sided. Therefore, we propose a method of functional synchronization analysis of brain regions. This method takes the EEG signal based on empirical modal decomposition (EMD) to obtain multiple intrinsic modal components (IMF) and inputs them into the Hilbert transform to obtain the instantaneous amplitude, and then calculates the amplitude locking value (ALV) to measure the synchronization relationship between all pairs of channels. The topological properties of the brain functional network are extracted to classify awake and fatigue states. The brain functional network is constructed based on the adjacency matrix of each waveform obtained from the ALV between all pairs of channels to realize the synchronization analysis between brain regions. Moreover, we achieved a satisfactory classification accuracy (82.84%) using the discriminative connection features in the Alpha band. In this study, we analyzed the functional network of ALV brain in fatigue and awake state, and the results showed that the connections between brain regions in fatigue state were significantly increased, and the connections between brain regions in the awake state were significantly decreased, and the information interaction between brain regions was more orderly and efficient.

A Novel Fatigue Driving State Recognition and Warning Method Based on EEG and EOG Signals

Traffic accidents are easily caused by tired driving. If the fatigue state of the driver can be identified in time and a corresponding early warning can be provided, then the occurrence of traffic accidents could be avoided to a large extent. At present, the recognition of fatigue driving states is mostly based on recognition accuracy. Fatigue state is currently recognized by combining different features, such as facial expressions, electroencephalogram (EEG) signals, yawning, and the percentage of eyelid closure over the pupil over time (PERCLoS). The combination of these features increases the recognition time and lacks real-time performance. In addition, some features will increase error in the recognition result, such as yawning frequently with the onset of a cold or frequent blinking with dry eyes. On the premise of ensuring the recognition accuracy and improving the realistic feasibility and real-time recognition performance of fatigue driving states, a fast support vector machine (FSVM) algorithm based on EEGs and electrooculograms (EOGs) is proposed to recognize fatigue driving states. First, the collected EEG and EOG modal data are preprocessed. Second, multiple features are extracted from the preprocessed EEGs and EOGs. Finally, FSVM is used to classify and recognize the data features to obtain the recognition result of the fatigue state. Based on the recognition results, this paper designs a fatigue driving early warning system based on Internet of Things (IoT) technology. When the driver shows symptoms of fatigue, the system not only sends a warning signal to the driver but also informs other nearby vehicles using this system through IoT technology and manages the operation background.

Evaluation of a Fast Test Based on Biometric Signals to Assess Mental Fatigue at the Workplace-A Pilot Study

Non-pathological mental fatigue is a recurring, but undesirable condition among people in the fields of office work, industry, and education. This type of mental fatigue can often lead to negative outcomes, such as performance reduction and cognitive impairment in education; loss of focus and burnout syndrome in office work; and accidents leading to injuries or death in the transportation and manufacturing industries. Reliable mental fatigue assessment tools are promising in the improvement of performance, mental health and safety of students and workers, and at the same time, in the reduction of risks, accidents and the associated economic loss (e.g., medical fees and equipment reparations). The analysis of biometric (brain, cardiac, skin conductance) signals has proven to be effective in discerning different stages of mental fatigue; however, many of the reported studies in the literature involve the use of long fatigue-inducing tests and subject-specific models in their methodologies. Recent trends in the modeling of mental fatigue suggest the usage of non subject-specific (general) classifiers and a time reduction of calibration procedures and experimental setups. In this study, the evaluation of a fast and short-calibration mental fatigue assessment tool based on biometric signals and inter-subject modeling, using multiple linear regression, is presented. The proposed tool does not require fatigue-inducing tests, which allows fast setup and implementation. Electroencephalography, photopletismography, electrodermal activity, and skin temperature from 17 subjects were recorded, using an OpenBCI helmet and an Empatica E4 wristband. Correlations to self-reported mental fatigue levels (using the fatigue assessment scale) were calculated to find the best mental fatigue predictors. Three-class mental fatigue models were evaluated, and the best model obtained an accuracy of 88% using three features, β/θ (C3), and the α/θ (O2 and C3) ratios, from one minute of electroencephalography measurements. The results from this pilot study show the feasibility and potential of short-calibration procedures and inter-subject classifiers in mental fatigue modeling, and will contribute to the use of wearable devices for the development of tools oriented to the well-being of workers and students, and also in daily living activities.

Reconstructing EOG From EEG Timeseries: A Spatial Filtering Approach

Unobtrusive mental state monitoring based on neurosphysiological signals has seen thriving developments over the past decade, with a wide area of applications, from rehabilitation to neuroergonomics and neuromarketing. Particularly, electroencephalography (EEG) and electrooculography (EOG) have been popular techniques to obtain cognitive-relevant biosignals. However, current wearable systems may still pose practical inconvenience, motivating further interest to integrate EOG+EEG recording into streamlined frontal-only sensor montages with sufficient signal fidelity. We propose, here, a spatial filtering approach to reliably extract EOG signals from a reduced set of frontal EEG electrodes, placed on non-hair-bearing (NHB) areas. Within a common signal analytic framework, two distinct schemes are examined. The one is based on standard linear least squares (LLS) and the other on Least Absolute Shrinkage and Selection Operator (LASSO). Both schemes are data-driven techniques, require a small amount of training data, and lead to reliable estimators of EOG activity from EEG signals. The LASSO-based technique, in addition, provides guidelines that generalize well across subjects. Using experimental data, we provide some empirical evidence that our estimators can replace the actual EOG signals in algorithmic pipelines that automatically detect oculographic events, like blinks and saccades.

Research on Driving Fatigue Alleviation Using Interesting Auditory Stimulation Based on VMD-MMSE

The accurate detection and alleviation of driving fatigue are of great significance to traffic safety. In this study, we tried to apply the modified multi-scale entropy (MMSE) approach, based on variational mode decomposition (VMD), to driving fatigue detection. Firstly, the VMD was used to decompose EEG into multiple intrinsic mode functions (IMFs), then the best IMFs and scale factors were selected using the least square method (LSM). Finally, the MMSE features were extracted. Compared with the traditional sample entropy (SampEn), the VMD-MMSE method can identify the characteristics of driving fatigue more effectively. The VMD-MMSE characteristics combined with a subjective questionnaire (SQ) were used to analyze the change trends of driving fatigue under two driving modes: normal driving mode and interesting auditory stimulation mode. The results show that the interesting auditory stimulation method adopted in this paper can effectively relieve driving fatigue. In addition, the interesting auditory stimulation method, which simply involves playing interesting auditory information on the vehicle-mounted player, can effectively relieve driving fatigue. Compared with traditional driving fatigue-relieving methods, such as sleeping and drinking coffee, this interesting auditory stimulation method can relieve fatigue in real-time when the driver is driving normally.

Transcranial Auricular Vagus Nerve Stimulation (taVNS) and Ear-EEG: Potential for Closed-Loop Portable Non-invasive Brain Stimulation

No matter how hard we concentrate, our attention fluctuates – a fact that greatly affects our success in completing a current task. Here, we review work from two methods that, in a closed-loop manner, have the potential to ameliorate these fluctuations. Ear-EEG can measure electric brain activity from areas in or around the ear, using small and thus portable hardware. It has been shown to capture the state of attention with high temporal resolution. Transcutaneous auricular vagus nerve stimulation (taVNS) comes with the same advantages (small and light) and critically current research suggests that it is possible to influence ongoing brain activity that has been linked to attention. Following the review of current work on ear-EEG and taVNS we suggest that a combination of the two methods in a closed-loop system could serve as a potential application to modulate attention.

Online Mental Fatigue Monitoring via Indirect Brain Dynamics Evaluation

Driver mental fatigue leads to thousands of traffic accidents. The increasing quality and availability of low-cost electroencephalogram (EEG) systems offer possibilities for practical fatigue monitoring. However, non-data-driven methods, designed for practical, complex situations, usually rely on handcrafted data statistics of EEG signals. To reduce human involvement, we introduce a data-driven methodology for online mental fatigue detection: self-weight ordinal regression (SWORE). Reaction time (RT), referring to the length of time people take to react to an emergency, is widely considered an objective behavioral measure for mental fatigue state. Since regression methods are sensitive to extreme RTs, we propose an indirect RT estimation based on preferences to explore the relationship between EEG and RT, which generalizes to any scenario when an objective fatigue indicator is available. In particular, SWORE evaluates the noisy EEG signals from multiple channels in terms of two states: shaking state and steady state. Modeling the shaking state can discriminate the reliable channels from the uninformative ones, while modeling the steady state can suppress the task-nonrelevant fluctuation within each channel. In addition, an online generalized Bayesian moment matching (online GBMM) algorithm is proposed to online-calibrate SWORE efficiently per participant. Experimental results with 40 participants show that SWORE can maximally achieve consistent with RT, demonstrating the feasibility and adaptability of our proposed framework in practical mental fatigue estimation.

A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals

Driver fatigue is the one of the main reasons of the traffic accidents. The human brain is a complex structure, whose function can be evaluated with electroencephalogram (EEG). Automated driver fatigue detection utilizing EEG decreases the incidence probability of related traffic accidents. Therefore, devising an appropriate feature extraction technique and selecting a competent classification method can be considered as the crucial part of the effective driver fatigue detection. Therefore, in this study, an EEG-based intelligent system was devised for driver fatigue detection. The proposed framework includes a new feature generation network, which is implemented by using texture descriptors, for fatigue detection. The proposed scheme contains pre-processing, feature generation, informative features selection and classification with shallow classifiers phases. In the pre-processing, discrete cosine transform and fast Fourier transform are used together. Moreover, dynamic center based binary pattern and multi threshold ternary pattern are utilized together to create a new feature generation network. To improve the detection performance, we utilized discrete wavelet transform as a pooling method, in which the functional brain network-based feature describing the relationship between fatigue and brain network organization. In the feature selection phase, a hybrid three layered feature selection method is presented, and benchmark classifiers are used in the classification phase to demonstrate the strength of the proposed method. In the experiments, the proposed framework achieved 97.29% classification accuracy for fatigue detection using EEG signals. This result reveals that the proposed framework can be utilized effectively for driver fatigue detection.

Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue

Fatigue driving has attracted a great deal of attention for its huge influence on automobile accidents. Recognizing driving fatigue provides a primary but significant way for addressing this problem. In this paper, we first conduct the simulated driving experiments to acquire the EEG signals in alert and fatigue states. Then, for multi-channel EEG signals without pre-processing, a novel rhythm-dependent multilayer brain network (RDMB network) is developed and analyzed for driving fatigue detection. We find that there exists a significant difference between alert and fatigue states from the view of network science. Further, key sub-RDMB network based on closeness centrality are extracted. We calculate six network measures from the key sub-RDMB network and construct feature vectors to classify the alert and fatigue states. The results show that our method can respectively achieve the average accuracy of 95.28% (with sample length of 5 s), 90.25% (2 s), and 87.69% (1 s), significantly higher than compared methods. All these validate the effectiveness of RDMB network for reliable driving fatigue detection via EEG.

Feature extraction of EEG signals based on functional data analysis and its application to recognition of driver fatigue state

OBJECTIVE

Our objective is to study how to obtain features which can reflect the continuity and internal dynamic changes of electroencephalography (EEG) signals and study an effective method for fatigued driving state recognition based on the obtained features.

APPROACH

A method of EEG signalfeature extraction based on functional data analysis is proposed. Combined with kernel principal component analysis method, the obtained features are applied to the recognition of driver fatigue state, and a corresponding recognition model of fatigued driving state is constructed.

MAIN RESULTS

The recognition model is tested on the real collected driver fatigue EEG signals by selecting a suitable classifier. The test results show that the proposed driver fatigue state recognition method has good recognition effect, especially on the classifier based on decision tree, with an average accuracy of 99.50%.

SIGNIFICANCE

The extracted features well reflect the continuityand internal dynamic changes of the EEG signals, and it is of great significance and application value to study an effective method of fatigued driver state recognition based on the features.

BCI-Based Rehabilitation on the Stroke in Sequela Stage

Background

Stroke is the leading cause of serious and long-term disability worldwide. Survivors may recover some motor functions after rehabilitation therapy. However, many stroke patients missed the best time period for recovery and entered into the sequela stage of chronic stroke.

Method

Studies have shown that motor imagery- (MI-) based brain-computer interface (BCI) has a positive effect on poststroke rehabilitation. This study used both virtual limbs and functional electrical stimulation (FES) as feedback to provide patients with a closed-loop sensorimotor integration for motor rehabilitation. An MI-based BCI system acquired, analyzed, and classified motor attempts from electroencephalogram (EEG) signals. The FES system would be activated if the BCI detected that the user was imagining wrist dorsiflexion on the instructed side of the body. Sixteen stroke patients in the sequela stage were randomly assigned to a BCI group and a control group. All of them participated in rehabilitation training for four weeks and were assessed by the Fugl-Meyer Assessment (FMA) of motor function.

Results

The average improvement score of the BCI group was 3.5, which was higher than that of the control group (0.9). The active EEG patterns of the four patients in the BCI group whose FMA scores increased gradually became centralized and shifted to sensorimotor areas and premotor areas throughout the study.

Conclusions

Study results showed evidence that patients in the BCI group achieved larger functional improvements than those in the control group and that the BCI-FES system is effective in restoring motor function to upper extremities in stroke patients. This study provides a more autonomous approach than traditional treatments used in stroke rehabilitation.

Review of semi-dry electrodes for EEG recording

Developing reliable and user-friendly electroencephalography (EEG) electrodes remains a challenge for emerging real-world EEG applications. Classic wet electrodes are the gold standard for recording EEG; however, they are difficult to implement and make users uncomfortable, thus severely restricting their widespread application in real-life scenarios. An alternative is dry electrodes, which do not require conductive gels or skin preparation. Despite their quick setup and improved user-friendliness, dry electrodes still have some inherent problems (invasive, relatively poor signal quality, or sensitivity to motion artifacts), which limit their practical utilization. In recent years, semi-dry electrodes, which require only a small amount of electrolyte fluid, have been successfully developed, combining the advantages of both wet and dry electrodes while addressing their respective drawbacks. Semi-dry electrodes can collect reliable EEG signals comparable to wet electrodes. Moreover, their setup is as fast and convenient similar to that of dry electrodes. Hence, semi-dry electrodes have shown tremendous application prospects for real-world EEG acquisition. Herein, we systematically summarize the development, evaluation methods, and practical design considerations of semi-dry electrodes. Some feasible suggestions and new ideas for the development of semi-dry electrodes have been presented. This review provides valuable technical support for the development of semi-dry electrodes toward emerging practical applications.

Instantaneous mental workload assessment using time-frequency analysis and semi-supervised learning

The real-time assessment of mental workload (MWL) is critical for development of intelligent human–machine cooperative systems in various safety–critical applications. Although data-driven machine learning (ML) approach has shown promise in MWL recognition, there is still difficulty in acquiring a sufficient number of labeled data to train the ML models. This paper proposes a semi-supervised extreme learning machine (SS-ELM) algorithm for MWL pattern classification requiring only a small number of labeled data. The measured data analysis results show that the proposed SS-ELM paradigm can effectively improve the accuracy and efficiency of MWL classification and thus provide a competitive ML approach to utilizing a large number of unlabeled data which are available in many real-world applications.

Between-Frequency Topographical and Dynamic High-Order Functional Connectivity for Driving Drowsiness Assessment

Previous studies exploring driving drowsiness utilized spectral power and functional connectivity without considering between-frequency and more complex synchronizations. To complement such lacks, we explored inter-regional synchronizations based on the topographical and dynamic properties between frequency bands using high-order functional connectivity (HOFC) and envelope correlation. We proposed the dynamic interactions of HOFC, associated-HOFC, and a global metric measuring the aggregated effect of the functional connectivity. The EEG dataset was collected from 30 healthy subjects, undergoing two driving sessions. The two-session setting was employed for evaluating the metric reliability across sessions. Based on the results, we observed reliably significant metric changes, mainly involving the alpha band. In HOFC_θα , HOFC_αβ , associated- HOFC_θα , and associated- HOFC_αβ , the connection-level metrics in frontal-central, central-central, and central-parietal/occipital areas were significantly increased, indicating a dominance in the central region. Similar results were also obtained in the HOFC_θαβ and aHOFC_θαβ . For dynamic-low-order-FC and dynamic-HOFC, the global metrics revealed a reliably significant increment in the alpha, theta-alpha, and alpha-beta bands. Modularity indexes of associated- HOFC_α and associated- HOFC_θα also exhibited reliably significant differences. This paper demonstrated that within-band and between-frequency topographical and dynamic FC can provide complementary information to the traditional individual-band LOFC for assessing driving drowsiness.

Accident Prediction System Based on Hidden Markov Model for Vehicular Ad-Hoc Network in Urban Environments

With the emergence of autonomous vehicles and internet of vehicles (IoV), future roads of smart cities will have a combination of autonomous and automated vehicles with regular vehicles that require human operators. To ensure the safety of the road commuters in such a network, it is imperative to enhance the performance of Advanced Driver Assistance Systems (ADAS). Real-time driving risk prediction is a fundamental part of an ADAS. Many driving risk prediction systems have been proposed. However, most of them are based only on vehicle’s velocity. But in most of the accident scenarios, other factors are also involved, such as weather conditions or driver fatigue. In this paper, we proposed an accident prediction system for Vehicular ad hoc networks (VANETs) in urban environments, in which we considered the crash risk as a latent variable that can be observed using multi-observation such as velocity, weather condition, risk location, nearby vehicles density and driver fatigue. A Hidden Markov Model (HMM) was used to model the correlation between these observations and the latent variable. Simulation results showed that the proposed system has a better performance in terms of sensitivity and precision compared to state of the art single factor schemes.