EEG-based driving fatigue detection using multilevel feature extraction and iterative hybrid feature selection

Ultra-Wide Band Radar Empowered Driver Drowsiness Detection with Convolutional Spatial Feature Engineering and Artificial Intelligence

Driving while drowsy poses significant risks, including reduced cognitive function and the potential for accidents, which can lead to severe consequences such as trauma, economic losses, injuries, or death. The use of artificial intelligence can enable effective detection of driver drowsiness, helping to prevent accidents and enhance driver performance. This research aims to address the crucial need for real-time and accurate drowsiness detection to mitigate the impact of fatigue-related accidents. Leveraging ultra-wideband radar data collected over five minutes, the dataset was segmented into one-minute chunks and transformed into grayscale images. Spatial features are retrieved from the images using a two-dimensional Convolutional Neural Network. Following that, these features were used to train and test multiple machine learning classifiers. The ensemble classifier RF-XGB-SVM, which combines Random Forest, XGBoost, and Support Vector Machine using a hard voting criterion, performed admirably with an accuracy of 96.6%. Additionally, the proposed approach was validated with a robust k-fold score of 97% and a standard deviation of 0.018, demonstrating significant results. The dataset is augmented using Generative Adversarial Networks, resulting in improved accuracies for all models. Among them, the RF-XGB-SVM model outperformed the rest with an accuracy score of 99.58%.

Fatigue at the wheel: A non-visual approach to truck driver fatigue detection by multi-feature fusion

BACKGROUND

Monitoring of long-haul truck driver fatigue state has attracted considerable interest. Conventional fatigue driving detection methods based on the physiological and visual features are scarcely applicable, due to the intrusiveness, reliability, and cost-effectiveness concerns.

METHODS

We elaborately developed a fatigue driving detection method by fusion of non-visual features derived from the customized wristbands, vehicle-mounted equipment, and trip logs. To capture the spatiotemporal information within the sequential data, the bidirectional long short-term memory network with attention mechanism was proposed to determine whether the truck driver was fatigued within a fine-grained episode of one minute. The model was validated using a natural driving dataset with nine truck drivers on real-world roads in Guiyang, China during June and July 2021.

RESULTS

Our approach yielded 99.21 %, 84.44 %, 82.01 %, 99.63 %, and 83.21 % in accuracy, precision, recall, specificity, and F1-score, respectively. Compared with the mainstream visual-based methods, our approach outperformed particularly in terms of precision and recall. Photoplethysmogram stood out as the most important feature for truck driver fatigue state detection. Vehicle load, driving forward angle, cumulative driving time, midnight, and recent working hours were found to be positively associated with the probability of fatigue driving, while the galvanic skin response, vehicle acceleration, current time, and recent rest hours had a negative relationship. Specifically, truck drivers were more likely to fatigue when driving at 20-40 km/h, braking abruptly at 5-10 m/s2, with vehicle loads over 70 tons, and driving more than 100 min consecutively.

CONCLUSIONS

Our study is among the first to harness the natural driving dataset to delve into the real-life fatigue pattern of long-haul truck drivers without disruptions on routine driving tasks. The proposed method holds pragmatic prospects by providing a privacy-preserving, robust, real-time, and non-intrusive technical pathway for truck driver fatigue monitoring.

Driving fatigue detection based on brain source activity and ARMA model

Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.

Study on the effect of human sympathetic nerve cold stimulation to relieve driving fatigue based on order recurrence plot

Driving fatigue is very likely to cause traffic accidents, seriously threatening the lives and properties of drivers. Therefore, accurate detection and effective mitigation of driving fatigue are crucial for ensuring the personal safety of drivers. This study proposes a method to relieve driving fatigue by properly reducing the temperature to stimulate the human sympathetic nerve. The method uses the intelligent cooling and blowing device on the car seat cushion to achieve cold stimulation of the sympathetic nerve of the driver by reducing the temperature of the driver's hip, back and neck, so as to increase the excitement of the sympathetic nerve, keep the driver alert and achieve the purpose of fighting driving fatigue. In view of the fact that the traditional fatigue detection method is easily affected by environmental factors and individual differences, this study uses the order recurrence plot (ORP) method to detect driving fatigue based on electroencephalogram (EEG) signals. The results show that ORP textures drawn by EEG signals of the two driving conditions (normal driving condition and sensory cold stimulation driving condition) are significantly different, and the quantization parameters determinism (DET) and average diagonal line length (DLL) values are significantly different. Cold stimulation of the subjects' hips, back and neck to alleviate driving fatigue was the best when the temperature was 21 °C. In addition, compared with the traditional methods of fatigue relief, the sensory cold stimulation method proposed in this study does not easily to produce tolerance and has no damage to the body.

Blink-related EEG activity measures cognitive load during proactive and reactive driving

Assessing drivers' cognitive load is crucial for driving safety in challenging situations. This research employed the occurrence of drivers' natural eye blinks as cues in continuously recorded EEG data to assess the cognitive workload while reactive or proactive driving. Twenty-eight participants performed either a lane-keeping task with varying levels of crosswind (reactive) or curve road (proactive). The blink event-related potentials (bERPs) and spectral perturbations (bERSPs) were analyzed to assess cognitive load variations. The study found that task load during reactive driving did not significantly impact bERPs or bERSPs, possibly due to enduring alertness for vehicle control. The proactive driving revealed significant differences in the occipital N1 component with task load, indicating the necessity to adapt the attentional resources allocation based on road demands. Also, increased steering complexity led to decreased frontal N2, parietal P3, occipital P2 amplitudes, and alpha power, requiring more cognitive resources for processing relevant information. Interestingly, the proactive and reactive driving scenarios demonstrated a significant interaction at the parietal P2 and occipital N1 for three difficulty levels. The study reveals that EEG measures related to natural eye blink behavior provide insights into the effect of cognitive load on different driving tasks, with implications for driver safety.

EEG feature selection method based on maximum information coefficient and quantum particle swarm

To reduce the dimensionality of EEG features and improve classification accuracy, we propose an improved hybrid feature selection method for EEG feature selection. First, MIC is used to remove irrelevant features and redundant features to reduce the search space of the second stage. QPSO is then used to optimize the feature in the second stage to obtain the optimal feature subset. Considering that both dimensionality and classification accuracy affect the performance of feature subsets, we design a new fitness function. Moreover, we optimize the parameters of the classifier while optimizing the feature subset to improve the classification accuracy and reduce the running time of the algorithm. Finally, experiments were performed on EEG and UCI datasets and compared with five existing feature selection methods. The results show that the feature subsets obtained by the proposed method have low dimensionality, high classification accuracy, and low computational complexity, which validates the effectiveness of the proposed method.

Driver drowsiness detection methods using EEG signals: a systematic review

Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.

Prediction of drowsiness using EEG signals in young Indonesian drivers

Indonesia is among the countries with the highest accident rates in the world. Fatigue and drowsiness are among the main causes of the increased risks of accidents in the road transport sector. Sleep-related factors (quality and quantity, time of day) and work-related factors significantly affect the development of fatigue. The EEG signal indicator is often referred to as the gold standard for measuring fatigue and drowsiness. However, previous studies focused primarily on the trends of EEG signals under certain conditions but overlooking the development of drowsiness indicators based on EEG signals. Furthermore, existing studies still do not agree on what parameters in the EEG signal indicator are best at detecting drowsiness. Thus, this study aims to design an EEG signal-based drowsiness indicator under simulated driving conditions. Drowsy drivers were monitored through EEG signal indicators and subjective assessments. The methods used in this study include statistical significance tests, logistic regression, and support vector machine. The results showed that sleep deprivation had a significant effect on increasing alpha, beta, and theta waves. In addition, driving duration significantly increased the theta power and all EEG ratios and decreased the beta power in the alert group. The ratio of (θ + α)/β and θ/β in the SD group also showed a considerable increase in the end of driving. Furthermore, sleep status and driving duration both influenced subjective sleepiness. EEG signals combined with sleep status and driving duration factors generated acceptable model accuracies (77.1% and 90.2% in training and testing, respectively), with 90.5% sensitivity and 90% specificity in data test. Support vector machine showed better classification than that of logistics regression, with the linear kernel as the best classifier. Theta power had the highest effect in the model compared with other EEG signals.

An improved multi-source domain adaptation network for inter-subject mental fatigue detection based on DANN

OBJECTIVES

Electroencephalogram (EEG) is often used to detect mental fatigue because of its real-time characteristic and objective nature. However, because of the individual variability of EEG among different individuals, tedious and time-consuming calibration sessions are needed.

METHODS

Therefore, we propose a multi-source domain adaptation network for inter-subject mental fatigue detection named FLDANN, which is short for focal loss based domain-adversarial training of neural network. As for mental state feature extraction, power spectrum density is extracted based on the Welch method from four sub-bands of EEG signals. The features of the source domain and target domain are fed into the FLDANN network. The contributions of FLDANN include: (1) It uses the idea of adversarial to reduce feature differences between the source and target domain. (2) A loss function named focal loss is used to assign weights to source and target domain samples.

RESULTS

The experiment result shows that when the number of the source domains increases, the classification accuracy of domain-adversarial training of neural network (DANN) gradually decreases and finally tends to be stable. The proposed method achieves an accuracy of 84.10% ± 8.75% on the SEED-VIG dataset and 65.42% ± 7.47% on the self-designed dataset. In addition, the proposed method is compared with other domain adaptation methods and the results show that the proposed method outperforms those state-of-the-art methods.

CONCLUSIONS

The result proves that the proposed method is able to solve the problem of individual differences across subjects and to solve the problem of low classification performance of multi-source domain transfer learning.

Automated prediction of COVID-19 mortality outcome using clinical and laboratory data based on hierarchical feature selection and random forest classifier

Early prediction of COVID-19 mortality outcome can decrease expiration risk by alerting healthcare personnel to assure efficient resource allocation and treatment planning. This study introduces a machine learning framework for the prediction of COVID-19 mortality using demographics, vital signs, and laboratory blood tests (complete blood count (CBC), coagulation, kidney, liver, blood gas, and general). 41 features from 244 COVID-19 patients were recorded on the first day of admission. In this study, first, the features in each of the eight categories were investigated. Afterward, features that have an area under the receiver operating characteristic curve (AUC) above 0.6 and the p-value criterion from the Wilcoxon rank-sum test below 0.005 were used as selected features for further analysis. Then five feature reduction methods, Forward Feature selection, minimum Redundancy Maximum Relevance, Relieff, Linear Discriminant Analysis, and Neighborhood Component Analysis were utilized to select the best combination of features. Finally, seven classifiers frameworks, random forest (RF), support vector machine, logistic regression (LR), K nearest neighbors, Artifical neural network, bagging, and boosting were used to predict the mortality outcome of COVID-19 patients. The results revealed that the combination of features in CBC and then vital signs had the highest mortality classification parameters, respectively. Furthermore, the RF classifier with hierarchical feature selection algorithms via Forward Feature selection had the highest classification power with an accuracy of 92.08 ± 2.56. Therefore, our proposed method can be confidently used as a valuable assistant prognostic tool to sieve patients with high mortality risks.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

Generalisable machine learning models trained on heart rate variability data to predict mental fatigue

A prolonged period of cognitive performance often leads to mental fatigue, a psychobiological state that increases the risk of injury and accidents. Previous studies have trained machine learning algorithms on Heart Rate Variability (HRV) data to detect fatigue in order to prevent its consequences. However, the results of these studies cannot be generalised because of various methodological issues including the use of only one type of cognitive task to induce fatigue which makes any predictions task-specific. In this study, we combined the datasets of three experiments each of which applied different cognitive tasks for fatigue induction and trained algorithms that detect fatigue and predict its severity. We also tested different time window lengths and compared algorithms trained on resting and task related data. We found that classification performance was best when the support vector classifier was trained on task related HRV calculated for a 5-min time window (AUC = 0.843, accuracy = 0.761). For the prediction of fatigue severity, CatBoost regression showed the best performance when trained on 3-min HRV data and self-reported measures (R² = 0.248, RMSE = 17.058). These results indicate that both the detection and prediction of fatigue based on HRV are effective when machine learning models are trained on heterogeneous, multi-task datasets.

Novel nested patch-based feature extraction model for automated Parkinson's Disease symptom classification using MRI images

OBJECTIVE

Parkinson's disease (PD) is a common neurological disorder with variable clinical manifestations and magnetic resonance imaging (MRI) findings. We propose a handcrafted image classification model that can accurately (i) classify different PD stages, (ii) detect comorbid dementia, and (iii) discriminate PD-related motor symptoms.

METHODS

Selected image datasets from three PD studies were used to develop the classification model. Our proposed novel automated system was developed in four phases: (i) texture features are extracted from the non-fixed size patches. In the feature extraction phase, a pyramid histogram-oriented gradient (PHOG) image descriptor is used. (ii) In the feature selection phase, four feature selectors: neighborhood component analysis (NCA), Chi2, minimum redundancy maximum relevancy (mRMR), and ReliefF are used to generate four feature vectors. (iii) Two classifiers: k-nearest neighbor (kNN) and support vector machine (SVM) are used in the classification step. A ten-fold cross-validation technique is used to validate the results. (iv) Eight predicted vectors are generated using four selected feature vectors and two classifiers. Finally, iterative majority voting (IMV) is used to attain general classification results. Therefore, this model is named nested patch-PHOG-multiple feature selectors and multiple classifiers-IMV (NP-PHOG-MFSMCIMV).

RESULTS

Our presented NP-PHOG-MFSMCIMV model achieved 99.22, 98.70, and 99.53% accuracies for the collected PD stages, PD dementia, and PD symptoms classification datasets, respectively.

SIGNIFICANCE

The obtained accuracies (over 98% for all states) demonstrated the performance of developed NP-PHOG-MFSMCIMV model in automated PD state classification.

A Virtual Reality and Online Learning Immersion Experience Evaluation Model Based on SVM and Wearable Recordings

The increasing development in the field of biosensing technologies makes it feasible to monitor students’ physiological signals in natural learning scenarios. With the rise of mobile learning, educators are attaching greater importance to the learning immersion experience of students, especially with the global background of COVID-19. However, traditional methods, such as questionnaires and scales, to evaluate the learning immersion experience are greatly influenced by individuals’ subjective factors. Herein, our research aims to explore the relationship and mechanism between human physiological recordings and learning immersion experiences to eliminate subjectivity as much as possible. We collected electroencephalogram and photoplethysmographic signals, as well as self-reports on the immersive experience of thirty-seven college students during virtual reality and online learning to form the fundamental feature set. Then, we proposed an evaluation model based on a support vector machine and got a precision accuracy of 89.72%. Our research results provide evidence supporting the possibility of predicting students’ learning immersion experience by their EEGs and PPGs.

Enhancing EEG-Based Mental Stress State Recognition Using an Improved Hybrid Feature Selection Algorithm

In real-life applications, electroencephalogram (EEG) signals for mental stress recognition require a conventional wearable device. This, in turn, requires an efficient number of EEG channels and an optimal feature set. This study aims to identify an optimal feature subset that can discriminate mental stress states while enhancing the overall classification performance. We extracted multi-domain features within the time domain, frequency domain, time-frequency domain, and network connectivity features to form a prominent feature vector space for stress. We then proposed a hybrid feature selection (FS) method using minimum redundancy maximum relevance with particle swarm optimization and support vector machines (mRMR-PSO-SVM) to select the optimal feature subset. The performance of the proposed method is evaluated and verified using four datasets, namely EDMSS, DEAP, SEED, and EDPMSC. To further consolidate, the effectiveness of the proposed method is compared with that of the state-of-the-art metaheuristic methods. The proposed model significantly reduced the features vector space by an average of 70% compared with the state-of-the-art methods while significantly increasing overall detection performance.

An Effective and Robust Approach Based on R-CNN+LSTM Model and NCAR Feature Selection for Ophthalmological Disease Detection from Fundus Images

Changes in and around anatomical structures such as blood vessels, optic disc, fovea, and macula can lead to ophthalmological diseases such as diabetic retinopathy, glaucoma, age-related macular degeneration (AMD), myopia, hypertension, and cataracts. If these diseases are not diagnosed early, they may cause partial or complete loss of vision in patients. Fundus imaging is the primary method used to diagnose ophthalmologic diseases. In this study, a powerful R-CNN+LSTM-based approach is proposed that automatically detects eight different ophthalmologic diseases from fundus images. Deep features were extracted from fundus images with the proposed R-CNN+LSTM structure. Among the deep features extracted, those with high representative power were selected with an approach called NCAR, which is a multilevel feature selection algorithm. In the classification phase, the SVM algorithm, which is a powerful classifier, was used. The proposed approach is evaluated on the eight-class ODIR dataset. The accuracy (main metric), sensitivity, specificity, and precision metrics were used for the performance evaluation of the proposed approach. Besides, the performance of the proposed approach was compared with the existing approaches using the ODIR dataset.