A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

A novel deep learning model based on transformer and cross modality attention for classification of sleep stages

The classification of sleep stages is crucial for gaining insights into an individual's sleep patterns and identifying potential health issues. Employing several important physiological channels in different views, each providing a distinct perspective on sleep patterns, can have a great impact on the efficiency of the classification models. In the context of neural networks and deep learning models, transformers are very effective, especially when dealing with time series data, and have shown remarkable compatibility with sequential data analysis as physiological channels. On the other hand, cross-modality attention by integrating information from multiple views of the data enables to capture relationships among different modalities, allowing models to selectively focus on relevant information from each modality. In this paper, we introduce a novel deep-learning model based on transformer encoder-decoder and cross-modal attention for sleep stage classification. The proposed model processes information from various physiological channels with different modalities using the Sleep Heart Health Study Dataset (SHHS) data and leverages transformer encoders for feature extraction and cross-modal attention for effective integration to feed into the transformer decoder. The combination of these elements increased the accuracy of the model up to 91.33% in classifying five classes of sleep stages. Empirical evaluations demonstrated the model's superior performance compared to standalone approaches and other state-of-the-art techniques, showcasing the potential of combining transformer and cross-modal attention for improved sleep stage classification.

Multi-Index Driver Drowsiness Detection Method Based on Driver's Facial Recognition Using Haar Features and Histograms of Oriented Gradients

It is estimated that 10% to 20% of road accidents are related to fatigue, with accidents caused by drowsiness up to twice as deadly as those caused by other factors. In order to reduce these numbers, strategies such as advertising campaigns, the implementation of driving recorders in vehicles used for road transport of goods and passengers, or the use of drowsiness detection systems in cars have been implemented. Within the scope of the latter area, the technologies used are diverse. They can be based on the measurement of signals such as steering wheel movement, vehicle position on the road, or driver monitoring. Driver monitoring is a technology that has been exploited little so far and can be implemented in many different approaches. This work addresses the evaluation of a multidimensional drowsiness index based on the recording of facial expressions, gaze direction, and head position and studies the feasibility of its implementation in a low-cost electronic package. Specifically, the aim is to determine the driver's state by monitoring their facial expressions, such as the frequency of blinking, yawning, eye-opening, gaze direction, and head position. For this purpose, an algorithm capable of detecting drowsiness has been developed. Two approaches are compared: Facial recognition based on Haar features and facial recognition based on Histograms of Oriented Gradients (HOG). The implementation has been carried out on a Raspberry Pi, a low-cost device that allows the creation of a prototype that can detect drowsiness and interact with peripherals such as cameras or speakers. The results show that the proposed multi-index methodology performs better in detecting drowsiness than algorithms based on one-index detection.

Wireless ear EEG to monitor drowsiness

Neural wearables can enable life-saving drowsiness and health monitoring for pilots and drivers. While existing in-cabin sensors may provide alerts, wearables can enable monitoring across more environments. Current neural wearables are promising but most require wet-electrodes and bulky electronics. This work showcases in-ear, dry-electrode earpieces used to monitor drowsiness with compact hardware. The employed system integrates additive-manufacturing for dry, user-generic earpieces, existing wireless electronics, and offline classification algorithms. Thirty-five hours of electrophysiological data were recorded across nine subjects performing drowsiness-inducing tasks. Three classifier models were trained with user-specific, leave-one-trial-out, and leave-one-user-out splits. The support-vector-machine classifier achieved an accuracy of 93.2% while evaluating users it has seen before and 93.3% when evaluating a never-before-seen user. These results demonstrate wireless, dry, user-generic earpieces used to classify drowsiness with comparable accuracies to existing state-of-the-art, wet electrode in-ear and scalp systems. Further, this work illustrates the feasibility of population-trained classification in future electrophysiological applications.

A Review of EEG-based Localization of Epileptic Seizure Foci: Common Points with Multimodal Fusion of Brain Data

Unexpected seizures significantly decrease the quality of life in epileptic patients. Seizure attacks are caused by hyperexcitability and anatomical lesions of special regions of the brain, and cognitive impairments and memory deficits are their most common concomitant effects. In addition to seizure reduction treatments, medical rehabilitation involving brain-computer interfaces and neurofeedback can improve cognition and quality of life in patients with focal epilepsy in most cases, in particular when resective epilepsy surgery has been considered treatment in drug-resistant epilepsy. Source estimation and precise localization of epileptic foci can improve such rehabilitation and treatment. Electroencephalography (EEG) monitoring and multimodal noninvasive neuroimaging techniques such as ictal/interictal single-photon emission computerized tomography (SPECT) imaging and structural magnetic resonance imaging are common practices for the localization of epileptic foci and have been studied in several kinds of researches. In this article, we review the most recent research on EEG-based localization of seizure foci and discuss various methods, their advantages, limitations, and challenges with a focus on model-based data processing and machine learning algorithms. In addition, we survey whether combined analysis of EEG monitoring and neuroimaging techniques, which is known as multimodal brain data fusion, can potentially increase the precision of the seizure foci localization. To this end, we further review and summarize the key parameters and challenges of processing, fusion, and analysis of multiple source data, in the framework of model-based signal processing, for the development of a multimodal brain data analyzing system. This article has the potential to be used as a valuable resource for neuroscience researchers for the development of EEG-based rehabilitation systems based on multimodal data analysis related to focal epilepsy.

Towards predicting PTSD symptom severity using portable EEG-derived biomarkers

Posttraumatic Stress Disorder (PTSD) is a heterogeneous mental health disorder that occurs following traumatic experience. Understanding its neurobiological basis is crucial to advance early diagnosis and treatment. Electroencephalography (EEG) can be used to explore the neurobiological basis of PTSD. However, only limited research has explored mobile EEG, which is important for scalability. This proof-of-concept study delves into mobile EEG-derived biomarkers for PTSD and their potential implications. Over four weeks, we measured PTSD symptoms using the PTSD checklist for DSM-5 (PCL-5) at multiple timepoints, and we recorded multiple EEG sessions from 21 individuals using a mobile EEG device. In total, we captured 38 EEG sessions, each comprising two recordings that lasted approximately 180 seconds, to evaluate reproducibility. Next, we extracted Shannon entropy, as a measure of the randomness or unpredictability of the signal and spectral power for the fronto-temporal regions of interest, including electrodes at AF3, AF4, T7, and T8 for each EEG recording session. We calculated the partial correlation between the EEG variables and PCL-5 measured closest to the EEG session, using age, sex, and the grouping variable 'batch' as covariates. We observed a significant negative correlation between Shannon entropy in fronto-temporal regions and PCL-5 scores. Specifically, this association was evident in the AF3 ( r = -0.456, FDR-corrected p = 0.01), AF4 ( r = -0.362, FDR-corrected p = 0.04), and T7 ( r = -0.472, FDR-corrected p = 0.01) regions. Additionally, we found a significant negative association between the alpha power estimated from AF4 and PCL-5 ( r =-0.429, FDR-corrected p =0.04). Our findings suggest that EEG data acquired using a mobile EEG device is associated with PTSD symptom severity, offering valuable insights into the neurobiological mechanisms underlying PTSD.

Efficient Generalized Electroencephalography-Based Drowsiness Detection Approach with Minimal Electrodes

Drowsiness is a main factor for various costly defects, even fatal accidents in areas such as construction, transportation, industry and medicine, due to the lack of monitoring vigilance in the mentioned areas. The implementation of a drowsiness detection system can greatly help to reduce the defects and accident rates by alerting individuals when they enter a drowsy state. This research proposes an electroencephalography (EEG)-based approach for detecting drowsiness. EEG signals are passed through a preprocessing chain composed of artifact removal and segmentation to ensure accurate detection followed by different feature extraction methods to extract the different features related to drowsiness. This work explores the use of various machine learning algorithms such as Support Vector Machine (SVM), the K nearest neighbor (KNN), the Naive Bayes (NB), the Decision Tree (DT), and the Multilayer Perceptron (MLP) to analyze EEG signals sourced from the DROZY database, carefully labeled into two distinct states of alertness (awake and drowsy). Segmentation into 10 s intervals ensures precise detection, while a relevant feature selection layer enhances accuracy and generalizability. The proposed approach achieves high accuracy rates of 99.84% and 96.4% for intra (subject by subject) and inter (cross-subject) modes, respectively. SVM emerges as the most effective model for drowsiness detection in the intra mode, while MLP demonstrates superior accuracy in the inter mode. This research offers a promising avenue for implementing proactive drowsiness detection systems to enhance occupational safety across various industries.

Individual Variability in Brain Connectivity Patterns and Driving-Fatigue Dynamics

Mental fatigue during driving poses significant risks to road safety, necessitating accurate assessment methods to mitigate potential hazards. This study explores the impact of individual variability in brain networks on driving fatigue assessment, hypothesizing that subject-specific connectivity patterns play a pivotal role in understanding fatigue dynamics. By conducting a linear regression analysis of subject-specific brain networks in different frequency bands, this research aims to elucidate the relationships between frequency-specific connectivity patterns and driving fatigue. As such, an EEG sustained driving simulation experiment was carried out, estimating individuals' brain networks using the Phase Lag Index (PLI) to capture shared connectivity patterns. The results unveiled notable variability in connectivity patterns across frequency bands, with the alpha band exhibiting heightened sensitivity to driving fatigue. Individualized connectivity analysis underscored the complexity of fatigue assessment and the potential for personalized approaches. These findings emphasize the importance of subject-specific brain networks in comprehending fatigue dynamics, while providing sensor space minimization, advocating for the development of efficient mobile sensor applications for real-time fatigue detection in driving scenarios.

Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

Driver Drowsiness Multi-Method Detection for Vehicles with Autonomous Driving Functions

The article outlines various approaches to developing a fuzzy decision algorithm designed for monitoring and issuing warnings about driver drowsiness. This algorithm is based on analyzing EOG (electrooculography) signals and eye state images with the aim of preventing accidents. The drowsiness warning system comprises key components that learn about, analyze and make decisions regarding the driver's alertness status. The outcomes of this analysis can then trigger warnings if the driver is identified as being in a drowsy state. Driver drowsiness is characterized by a gradual decline in attention to the road and traffic, diminishing driving skills and an increase in reaction time, all contributing to a higher risk of accidents. In cases where the driver does not respond to the warnings, the ADAS (advanced driver assistance systems) system should intervene, assuming control of the vehicle's commands.

AS, Hatlestad-Hall C, Yazidi A, Hammer HL, Hebold Haraldsen IRJ. Introducing Region Based Pooling for handling a varied number of EEG channels for deep learning models

Introduction

A challenge when applying an artificial intelligence (AI) deep learning (DL) approach to novel electroencephalography (EEG) data, is the DL architecture's lack of adaptability to changing numbers of EEG channels. That is, the number of channels cannot vary neither in the training data, nor upon deployment. Such highly specific hardware constraints put major limitations on the clinical usability and scalability of the DL models.

Methods

In this work, we propose a technique for handling such varied numbers of EEG channels by splitting the EEG montages into distinct regions and merge the channels within the same region to a region representation. The solution is termed Region Based Pooling (RBP). The procedure of splitting the montage into regions is performed repeatedly with different region configurations, to minimize potential loss of information. As RBP maps a varied number of EEG channels to a fixed number of region representations, both current and future DL architectures may apply RBP with ease. To demonstrate and evaluate the adequacy of RBP to handle a varied number of EEG channels, sex classification based solely on EEG was used as a test example. The DL models were trained on 129 channels, and tested on 32, 65, and 129-channels versions of the data using the same channel positions scheme. The baselines for comparison were zero-filling the missing channels and applying spherical spline interpolation. The performances were estimated using 5-fold cross validation.

Results

For the 32-channel system version, the mean AUC values across the folds were: RBP (93.34%), spherical spline interpolation (93.36%), and zero-filling (76.82%). Similarly, on the 65-channel system version, the performances were: RBP (93.66%), spherical spline interpolation (93.50%), and zero-filling (85.58%). Finally, the 129-channel system version produced the following results: RBP (94.68%), spherical spline interpolation (93.86%), and zero-filling (91.92%).

Conclusion

In conclusion, RBP obtained similar results to spherical spline interpolation, and superior results to zero-filling. We encourage further research and development of DL models in the cross-dataset setting, including the use of methods such as RBP and spherical spline interpolation to handle a varied number of EEG channels.

Point of Care Testing (POCT) in Psychopathology Using Fractal Analysis and Hilbert Huang Transform of Electroencephalogram (EEG)

Research has shown that relying only on self-reports for diagnosing psychiatric disorders does not yield accurate results at all times. The advances of technology as well as artificial intelligence and other machine learning algorithms have allowed the introduction of point of care testing (POCT) including EEG characterization and correlations with possible psychopathology. Nonlinear methods of EEG analysis have significant advantages over linear methods. Empirical mode decomposition (EMD) is a reliable nonlinear method of EEG pre-processing. In this chapter, we compare two existing EEG complexity measures - Higuchi fractal dimension (HFD) and sample entropy (SE), with our newly proposed method using Higuchi fractal dimension from the Hilbert Huang transform (HFD-HHT). We present an example using the three complexity measures on a 2-minute EEG recorded from a healthy 20-year-old male after signal pre-processing. Furthermore, we showed the usefulness of these complexity measures in the classification of major depressive disorder (MDD) with healthy controls. Our study is in line with previous research and has shown an increase in HFD and SE values in the full, alpha and beta frequency bands suggestive of an increase in EEG irregularity. Moreover, the HFD-HHT values decreased in those three bands for majority of electrodes which is suggestive of a decrease in irregularity in the frequency-time domain. We conclude that all three complexity measures can be vital features useful for EEG analysis which could be incorporated in POCT systems.

Physiological measurements for driving drowsiness: A comparative study of multi-modality feature fusion and selection

A large number of traffic accidents were caused by drowsiness while driving. In-vehicle alert system based on physiological signals was one of the most promising solutions to monitor driving fatigue. However, different physiological modalities can be used, and many relative studies compared different modalities without considering the implementation feasibility of portable or wearable devices. Moreover, evaluations of each modality in previous studies were based on inconsistent choices of fatigue label and signal features, making it hard to compare the results of different studies. Therefore, the modality comparison and fusion for continuous drowsiness estimation while driving was still unclear. This work sought to comprehensively compare widely-used physiological modalities, including forehead electroencephalogram (EEG), electrooculogram (EOG), R-R intervals (RRI) and breath, in a hardware setting feasible for portable or wearable devices to monitor driving fatigue. Moreover, a more general conclusion on modality comparison and fusion was reached based on the regression of features or their combinations and the awake-to-drowsy transition. Finally, the feature subset of fused modalities was produced by feature selection method, to select the optimal feature combination and reduce computation consumption. Considering practical feasibility, the most effective combination with the highest correlation coefficient was using forehead EEG or EOG, along with RRI and RRI-derived breath. If more comfort and convenience was required, the combination of RRI and RRI-derived breath was also promising.

Neurophysiological mental fatigue assessment for developing user-centered Artificial Intelligence as a solution for autonomous driving

The human factor plays a key role in the automotive field since most accidents are due to drivers' unsafe and risky behaviors. The industry is now pursuing two main solutions to deal with this concern: in the short term, there is the development of systems monitoring drivers' psychophysical states, such as inattention and fatigue, and in the medium-long term, there is the development of fully autonomous driving. This second solution is promoted by recent technological progress in terms of Artificial Intelligence and sensing systems aimed at making vehicles more and more accurately aware of their "surroundings." However, even with an autonomous vehicle, the driver should be able to take control of the vehicle when needed, especially during the current transition from the lower (SAE < 3) to the highest level (SAE = 5) of autonomous driving. In this scenario, the vehicle has to be aware not only of its "surroundings" but also of the driver's psychophysical state, i.e., a user-centered Artificial Intelligence. The neurophysiological approach is one the most effective in detecting improper mental states. This is particularly true if considering that the more automatic the driving will be, the less available the vehicular data related to the driver's driving style. The present study aimed at employing a holistic approach, considering simultaneously several neurophysiological parameters, in particular, electroencephalographic, electrooculographic, photopletismographic, and electrodermal activity data to assess the driver's mental fatigue in real time and to detect the onset of fatigue increasing. This would ideally work as an information/trigger channel for the vehicle AI. In all, 26 professional drivers were engaged in a 45-min-lasting realistic driving task in simulated conditions, during which the previously listed biosignals were recorded. Behavioral (reaction times) and subjective measures were also collected to validate the experimental design and to support the neurophysiological results discussion. Results showed that the most sensitive and timely parameters were those related to brain activity. To a lesser extent, those related to ocular parameters were also sensitive to the onset of mental fatigue, but with a delayed effect. The other investigated parameters did not significantly change during the experimental session.

Entropy-Based Machine Learning Model for Fast Diagnosis and Monitoring of Parkinson's Disease

This study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) using rest-state EEG signals (rs-EEG) from 20 PD subjects and 20 normal control (NC) subjects at a sampling rate of 128 Hz. Based on the comparative analysis of the effectiveness of entropy calculation methods, fuzzy entropy showed the best results in diagnosing and monitoring PD using rs-EEG, with classification accuracy (ARKF) of ~99.9%. The most important frequency range of rs-EEG for PD-based diagnostics lies in the range of 0-4 Hz, and the most informative signals were mainly received from the right hemisphere of the head. It was also found that ARKF significantly decreased as the length of rs-EEG segments decreased from 1000 to 150 samples. Using a procedure for selecting the most informative features, it was possible to reduce the computational costs of classification by 11 times, while maintaining an ARKF ~99.9%. The proposed method can be used in the healthcare internet of things (H-IoT), where low-performance edge devices can implement ML sensors to enhance human resilience to PD.

Multimodal Approach for Pilot Mental State Detection Based on EEG

The safety of flight operations depends on the cognitive abilities of pilots. In recent years, there has been growing concern about potential accidents caused by the declining mental states of pilots. We have developed a novel multimodal approach for mental state detection in pilots using electroencephalography (EEG) signals. Our approach includes an advanced automated preprocessing pipeline to remove artefacts from the EEG data, a feature extraction method based on Riemannian geometry analysis of the cleaned EEG data, and a hybrid ensemble learning technique that combines the results of several machine learning classifiers. The proposed approach provides improved accuracy compared to existing methods, achieving an accuracy of 86% when tested on cleaned EEG data. The EEG dataset was collected from 18 pilots who participated in flight experiments and publicly released at NASA's open portal. This study presents a reliable and efficient solution for detecting mental states in pilots and highlights the potential of EEG signals and ensemble learning algorithms in developing cognitive cockpit systems. The use of an automated preprocessing pipeline, feature extraction method based on Riemannian geometry analysis, and hybrid ensemble learning technique set this work apart from previous efforts in the field and demonstrates the innovative nature of the proposed approach.

A Bio-Inspired Chaos Sensor Model Based on the Perceptron Neural Network: Machine Learning Concept and Application for Computational Neuro-Science

The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2~0.9. The Hindmarsh-Rose spike model was used to generate time series of spike intervals, and datasets for training and testing the perceptron. The selection of the hyperparameters of the perceptron model and the estimation of the sensor accuracy were performed using the K-block cross-validation method. Even for a hidden layer with one neuron, the model approximates the fuzzy entropy with good results and the metric R2~0.5 ÷ 0.8. In a simplified model with one neuron and equal weights in the first layer, the principle of approximation is based on the linear transformation of the average value of the time series into the entropy value. An example of using the chaos sensor on spike train of action potential recordings from the L5 dorsal rootlet of rat is provided. The bio-inspired chaos sensor model based on an ensemble of neurons is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing. The study will be useful for specialists in the field of computational neuroscience, and also to create humanoid and animal robots, and bio-robots with limited resources.

Influence of Multimodal Emotional Stimulations on Brain Activity: An Electroencephalographic Study

This study aimed to reveal the influence of emotional valence and sensory modality on neural activity in response to multimodal emotional stimuli using scalp EEG. In this study, 20 healthy participants completed the emotional multimodal stimulation experiment for three stimulus modalities (audio, visual, and audio-visual), all of which are from the same video source with two emotional components (pleasure or unpleasure), and EEG data were collected using six experimental conditions and one resting state. We analyzed power spectral density (PSD) and event-related potential (ERP) components in response to multimodal emotional stimuli, for spectral and temporal analysis. PSD results showed that the single modality (audio only/visual only) emotional stimulation PSD differed from multi-modality (audio-visual) in a wide brain and band range due to the changes in modality and not from the changes in emotional degree. The most pronounced N200-to-P300 potential shifts occurred in monomodal rather than multimodal emotional stimulations. This study suggests that emotional saliency and sensory processing efficiency perform a significant role in shaping neural activity during multimodal emotional stimulation, with the sensory modality being more influential in PSD. These findings contribute to our understanding of the neural mechanisms involved in multimodal emotional stimulation.

Assessing the development of mental fatigue during simulated flights with concurrent EEG-fNIRS measurement

Mental fatigue (MF) can impair pilots' performance and reactions to unforeseen events and is therefore an important concept within aviation. The physiological measurement of MF, especially with EEG and, in recent years, fNIRS, has gained much attention. However, a systematic investigation and comparison of the measurements is seldomly done. We induced MF via time on task during a 90-min simulated flight task and collected concurrent EEG-fNIRS, performance and self-report data from 31 participants. While their subjective MF increased linearly, the participants were able to keep their performance stable over the course of the experiment. EEG data showed an early increase and levelling in parietal alpha power and a slower, but steady increase in frontal theta power. No consistent trend could be observed in the fNIRS data. Thus, more research on fNIRS is needed to understand its possibilities and limits for MF assessment, and a combination with EEG is advisable to compare and validate results. Until then, EEG remains the better choice for continuous MF assessment in cockpit applications because of its high sensitivity to a transition from alert to fatigued, even before performance is impaired.

Automatic Identification of Children with ADHD from EEG Brain Waves

EEG (electroencephalogram) signals could be used reliably to extract critical information regarding ADHD (attention deficit hyperactivity disorder), a childhood neurodevelopmental disorder. The early detection of ADHD is important to lessen the development of this disorder and reduce its long-term impact. This study aimed to develop a computer algorithm to identify children with ADHD automatically from the characteristic brain waves. An EEG machine learning pipeline is presented here, including signal preprocessing and data preparation steps, with thorough explanations and rationale. A large public dataset of 120 children was selected, containing large variability and minimal measurement bias in data collection and reproducible child-friendly visual attentional tasks. Unlike other studies, EEG linear features were extracted to train a Gaussian SVM-based model from only the first four sub-bands of EEG. This eliminates signals more than 30 Hz, thus reducing the computational load for model training while keeping mean accuracy of ~94%. We also performed rigorous validation (obtained 93.2% and 94.2% accuracy, respectively, for holdout and 10-fold cross-validation) to ensure that the developed model is minimally impacted by bias and overfitting that commonly appear in the ML pipeline. These performance metrics indicate the ability to automatically identify children with ADHD from a local clinical setting and provide a baseline for further clinical evaluation and timely therapeutic attempts.

EEG microstate changes according to mental fatigue induced by aircraft piloting simulation: An exploratory study

BACKGROUND

A continuous flight task load can induce fatigue and lead to changes in electroencephalography (EEG). EEG microstates can reflect the activities of large-scale neural networks during mental fatigue. This exploratory experiment explored the effects of mental fatigue induced by continuous simulated flight multitasking on EEG microstate indices.

METHODS

Twenty-four participants performed continuous 2-hour aircraft piloting simulation while EEG were recorded. The Stanford sleepiness scale (SSS) and critical flicker fusion frequency (CFF) were measured before and after the task. Microstate analysis was applied to EEG. Four microstate classes (A-D) were identified during the pre-task, post-task, beginning, and end phases. The effects of mental fatigue were analyzed.

RESULTS

Compared with the pre-task, the post-task had a higher global explained variance (GEV) and time parameters of class C but lower occurrence and coverage of class D. The end had a higher GEV but lower duration and coverage of class D than at the beginning. After 2 h of multitasking, the transition probability between A and D, and between B and D decreased but between A and C increased. Subjective fatigue scores were negatively correlated with occurrence and coverage of class D. Task performance was negatively correlated with duration and coverage of class C but positively correlated with duration and occurrence of class B.

CONCLUSION

Time parameters and transition probability of EEG microstates can detect mental fatigue induced by continuous aircraft piloting simulation. The global brain network activation of mental fatigue can be detected by EEG microstates that can evaluate flight fatigue.

Designing an Embedded Feature Selection Algorithm for a Drowsiness Detector Model Based on Electroencephalogram Data

Driver fatigue reduces the safety of traditional driving and limits the widespread adoption of self-driving cars; hence, the monitoring and early detection of drivers' drowsiness plays a key role in driving automation. When representing the drowsiness indicators as large feature vectors, fitting a machine learning model to the problem becomes challenging, and the problem's perspicuity decreases, making dimensionality reduction crucial in practice. For this reason, we propose an embedded feature selection algorithm that can be later utilized as a building block in the system development of a neural network-based drowsiness detector. We have adopted a technique: a so-called Feature Prune Layer is placed in front of the first layer in the architecture; as a result, its weights change regarding the importance of the corresponding input features and are deleted iteratively until the desired number is reached. We test the algorithm on EEG data, as it is one of the best indicators of drowsiness based on the literature. The proposed FS algorithm is able to reduce the original feature set by 95% with only 1% degradation in precision, while the precision increases by 1.5% and 2.7% respectively when selecting the top 10% and top 20% of the initial features. Moreover, the proposed method outperforms the widely popular Principal Component Analysis and the Chi-squared test when reducing the original feature set by 95%: it achieves 24.3% and 3.2% higher precision respectively.

On the Influence of Aging on Classification Performance in the Visual EEG Oddball Paradigm Using Statistical and Temporal Features

The utilization of a non-invasive electroencephalogram (EEG) as an input sensor is a common approach in the field of the brain-computer interfaces (BCI). However, the collected EEG data pose many challenges, one of which may be the age-related variability of event-related potentials (ERPs), which are often used as primary EEG BCI signal features. To assess the potential effects of aging, a sample of 27 young and 43 older healthy individuals participated in a visual oddball study, in which they passively viewed frequent stimuli among randomly occurring rare stimuli while being recorded with a 32-channel EEG set. Two types of EEG datasets were created to train the classifiers, one consisting of amplitude and spectral features in time and another with extracted time-independent statistical ERP features. Among the nine classifiers tested, linear classifiers performed best. Furthermore, we show that classification performance differs between dataset types. When temporal features were used, maximum individuals' performance scores were higher, had lower variance, and were less affected overall by within-class differences such as age. Finally, we found that the effect of aging on classification performance depends on the classifier and its internal feature ranking. Accordingly, performance will differ if the model favors features with large within-class differences. With this in mind, care must be taken in feature extraction and selection to find the correct features and consequently avoid potential age-related performance degradation in practice.

Making a case for endovascular approaches for neural recording and stimulation

There are many electrode types for recording and stimulating neural tissue, most of which necessitate direct contact with the target tissue. These electrodes range from large, scalp electrodes which are used to non-invasively record averaged, low frequency electrical signals from large areas/volumes of the brain, to penetrating microelectrodes which are implanted directly into neural tissue and interface with one or a few neurons. With the exception of scalp electrodes (which provide very low-resolution recordings), each of these electrodes requires a highly invasive, open brain surgical procedure for implantation, which is accompanied by significant risk to the patient. To mitigate this risk, a minimally invasive endovascular approach can be used. Several types of endovascular electrodes have been developed to be delivered into the blood vessels in the brain via a standard catheterization procedure. In this review, the existing body of research on the development and application of endovascular electrodes is presented. The capabilities of each of these endovascular electrodes is compared to commonly used direct-contact electrodes to demonstrate the relative efficacy of the devices. Potential clinical applications of endovascular recording and stimulation and the advantages of endovascular versus direct-contact approaches are presented.

An Efficient Machine Learning-Based Emotional Valence Recognition Approach Towards Wearable EEG

Emotion artificial intelligence (AI) is being increasingly adopted in several industries such as healthcare and education. Facial expressions and tone of speech have been previously considered for emotion recognition, yet they have the drawback of being easily manipulated by subjects to mask their true emotions. Electroencephalography (EEG) has emerged as a reliable and cost-effective method to detect true human emotions. Recently, huge research effort has been put to develop efficient wearable EEG devices to be used by consumers in out of the lab scenarios. In this work, a subject-dependent emotional valence recognition method is implemented that is intended for utilization in emotion AI applications. Time and frequency features were computed from a single time series derived from the Fp1 and Fp2 channels. Several analyses were performed on the strongest valence emotions to determine the most relevant features, frequency bands, and EEG timeslots using the benchmark DEAP dataset. Binary classification experiments resulted in an accuracy of 97.42% using the alpha band, by that outperforming several approaches from literature by ~3-22%. Multiclass classification gave an accuracy of 95.0%. Feature computation and classification required less than 0.1 s. The proposed method thus has the advantage of reduced computational complexity as, unlike most methods in the literature, only two EEG channels were considered. In addition, minimal features concluded from the thorough analyses conducted in this study were used to achieve state-of-the-art performance. The implemented EEG emotion recognition method thus has the merits of being reliable and easily reproducible, making it well-suited for wearable EEG devices.

STGATE: Spatial-temporal graph attention network with a transformer encoder for EEG-based emotion recognition

Electroencephalogram (EEG) is a crucial and widely utilized technique in neuroscience research. In this paper, we introduce a novel graph neural network called the spatial-temporal graph attention network with a transformer encoder (STGATE) to learn graph representations of emotion EEG signals and improve emotion recognition performance. In STGATE, a transformer-encoder is applied for capturing time-frequency features which are fed into a spatial-temporal graph attention for emotion classification. Using a dynamic adjacency matrix, the proposed STGATE adaptively learns intrinsic connections between different EEG channels. To evaluate the cross-subject emotion recognition performance, leave-one-subject-out experiments are carried out on three public emotion recognition datasets, i.e., SEED, SEED-IV, and DREAMER. The proposed STGATE model achieved a state-of-the-art EEG-based emotion recognition performance accuracy of 90.37% in SEED, 76.43% in SEED-IV, and 76.35% in DREAMER dataset, respectively. The experiments demonstrated the effectiveness of the proposed STGATE model for cross-subject EEG emotion recognition and its potential for graph-based neuroscience research.

Supervised Machine Learning and Deep Learning Techniques for Epileptic Seizure Recognition Using EEG Signals-A Systematic Literature Review

Electroencephalography (EEG) is a complicated, non-stationary signal that requires extensive preprocessing and feature extraction approaches to be accurately analyzed. In recent times, Deep learning (DL) has shown great promise in exploiting the characteristics of EEG signals as it can learn relevant features from raw data autonomously. Although studies involving DL have become more common in the last two years, the topic of whether DL truly delivers advantages over conventional Machine learning (ML) methodologies remains unsettled. This study aims to present a detailed overview of the main challenges in the field of seizure detection, prediction, and classification utilizing EEG data, and the approaches taken to solve them using ML and DL methods. A systematic review was conducted surveying peer-reviewed publications published between 2017 and 16 July 2022 using two scientific databases (Web of Science and Scopus) totaling 6822 references after discarding duplicate publications. Whereas 2262 articles were screened based on the title, abstract, and keywords, only 214 were eligible for full-text assessment. A total of 91 papers have been included in this survey after meeting the eligible inclusion and exclusion criteria. The most significant findings from the review are summarized, and several important concepts involving ML and DL for seizure detection, prediction, and classification are discussed in further depth. This review aims to learn more about the different approaches for identifying different types and stages of epileptic seizures, which may then be employed to enhance the lives of epileptic patients in the future, as well as aid experts in the field.

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

In human-robot collaboration scenarios with shared workspaces, a highly desired performance boost is offset by high requirements for human safety, limiting speed and torque of the robot drives to levels which cannot harm the human body. Especially for complex tasks with flexible human behavior, it becomes vital to maintain safe working distances and coordinate tasks efficiently. An established approach in this regard is reactive servo in response to the current human pose. However, such an approach does not exploit expectations of the human's behavior and can therefore fail to react to fast human motions in time. To adapt the robot's behavior as soon as possible, predicting human intention early becomes a factor which is vital but hard to achieve. Here, we employ a recently developed type of brain-computer interface (BCI) which can detect the focus of the human's overt attention as a predictor for impending action. In contrast to other types of BCI, direct projection of stimuli onto the workspace facilitates a seamless integration in workflows. Moreover, we demonstrate how the signal-to-noise ratio of the brain response can be used to adjust the velocity of the robot movements to the vigilance or alertness level of the human. Analyzing this adaptive system with respect to performance and safety margins in a physical robot experiment, we found the proposed method could improve both collaboration efficiency and safety distance.

Comparison of Eye and Face Features on Drowsiness Analysis

Drowsiness is one of the leading causes of traffic accidents. For those who operate large machinery or motor vehicles, incidents due to lack of sleep can cause property damage and sometimes lead to grave consequences of injuries and fatality. This study aims to design learning models to recognize drowsiness through human facial features. In addition, this work analyzes the attentions of individual neurons in the learning model to understand how neural networks interpret drowsiness. For this analysis, gradient-weighted class activation mapping (Grad-CAM) is implemented in the neural networks to display the attention of neurons. The eye and face images are processed separately to the model for the training process. The results initially show that better results can be obtained by delivering eye images alone. The effect of Grad-CAM is also more reasonable using eye images alone. Furthermore, this work proposed a feature analysis method, K-nearest neighbors Sigma (KNN-Sigma), to estimate the homogeneous concentration and heterogeneous separation of the extracted features. In the end, we found that the fusion of face and eye signals gave the best results for recognition accuracy and KNN-sigma. The area under the curve (AUC) of using face, eye, and fusion images are 0.814, 0.897, and 0.935, respectively.

Evaluation of Machine Learning Algorithms for Classification of EEG Signals

In brain–computer interfaces (BCIs), it is crucial to process brain signals to improve the accuracy of the classification of motor movements. Machine learning (ML) algorithms such as artificial neural networks (ANNs), linear discriminant analysis (LDA), decision tree (D.T.), K-nearest neighbor (KNN), naive Bayes (N.B.), and support vector machine (SVM) have made significant progress in classification issues. This paper aims to present a signal processing analysis of electroencephalographic (EEG) signals among different feature extraction techniques to train selected classification algorithms to classify signals related to motor movements. The motor movements considered are related to the left hand, right hand, both fists, feet, and relaxation, making this a multiclass problem. In this study, nine ML algorithms were trained with a dataset created by the feature extraction of EEG signals.The EEG signals of 30 Physionet subjects were used to create a dataset related to movement. We used electrodes C3, C1, CZ, C2, and C4 according to the standard 10-10 placement. Then, we extracted the epochs of the EEG signals and applied tone, amplitude levels, and statistical techniques to obtain the set of features. LabVIEW™2015 version custom applications were used for reading the EEG signals; for channel selection, noise filtering, band selection, and feature extraction operations; and for creating the dataset. MATLAB 2021a was used for training, testing, and evaluating the performance metrics of the ML algorithms. In this study, the model of Medium-ANN achieved the best performance, with an AUC average of 0.9998, Cohen’s Kappa coefficient of 0.9552, a Matthews correlation coefficient of 0.9819, and a loss of 0.0147. These findings suggest the applicability of our approach to different scenarios, such as implementing robotic prostheses, where the use of superficial features is an acceptable option when resources are limited, as in embedded systems or edge computing devices.

Drowsiness Detection Using Ocular Indices from EEG Signal

Drowsiness is one of the main causes of road accidents and endangers the lives of road users. Recently, there has been considerable interest in utilizing features extracted from electroencephalography (EEG) signals to detect driver drowsiness. However, in most of the work performed in this area, the eyeblink or ocular artifacts present in EEG signals are considered noise and are removed during the preprocessing stage. In this study, we examined the possibility of extracting features from the EEG ocular artifacts themselves to perform classification between alert and drowsy states. In this study, we used the BLINKER algorithm to extract 25 blink-related features from a public dataset comprising raw EEG signals collected from 12 participants. Different machine learning classification models, including the decision tree, the support vector machine (SVM), the K-nearest neighbor (KNN) method, and the bagged and boosted tree models, were trained based on the seven selected features. These models were further optimized to improve their performance. We were able to show that features from EEG ocular artifacts are able to classify drowsy and alert states, with the optimized ensemble-boosted trees yielding the highest accuracy of 91.10% among all classic machine learning models.

EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications

Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called “MDrow index”, was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).

Assessment of Combination of Automated Pupillometry and Heart Rate Variability to Detect Driving Fatigue

Objectives

Approximately 20~30% of all traffic accidents are caused by fatigue driving. However, limited practicability remains a barrier for the real application of available techniques to detect driving fatigue. Use of pupillary light reflex (PLR) may be potentially effective for driving fatigue detection.

Methods

A 90 min monotonous simulated driving task was utilized to induce driving fatigue. During the task, PLR measurements were performed at baseline and at an interval of 30 min. Subjective rating scales, heart rate variability (HRV) were monitored simultaneously.

Results

Thirty-two healthy volunteers in China participated in our study. Based on the results of subjective evaluation and behavioral performances, driving fatigue was verified to be successfully induced by a simulated driving task. Significant variations of PLR and HRV parameters were observed, which also showed significant relevance with the change in Karolinska Sleepiness Scale at several timepoints (|r| = 0.55 ~ 0.72, P < 0.001). Furthermore, PLR variations had excellent ability to detect driving fatigue with high sensitivity and specificity, of which maximum constriction velocity variations achieved a sensitivity of 85.00% and specificity of 72.34% for driving fatigue detection, vs. 82.50 and 78.72% with a combination of HRV variations, a nonsignificant difference (AUC = 0.835, 0.872, P > 0.05).

Conclusions

Pupillary light reflex variation may be a potential indicator in the detection of driving fatigue, achieving a comparative performance compared with the combination with heart rate variability. Further work may be involved in developing a commercialized driving fatigue detection system based on pupillary parameters.

Naturalistic viewing conditions can increase task engagement and aesthetic preference but have only minimal impact on EEG quality

Free gaze and moving images are typically avoided in EEG experiments due to the expected generation of artifacts and noise. Yet for a growing number of research questions, loosening these rigorous restrictions would be beneficial. Among these is research on visual aesthetic experiences, which often involve open-ended exploration of highly variable stimuli. Here we systematically compare the effect of conservative vs. more liberal experimental settings on various measures of behavior, brain activity and physiology in an aesthetic rating task. Our primary aim was to assess EEG signal quality. 43 participants either maintained fixation or were allowed to gaze freely, and viewed either static images or dynamic (video) stimuli consisting of dance performances or nature scenes. A passive auditory background task (auditory steady-state response; ASSR) was added as a proxy measure for overall EEG recording quality. We recorded EEG, ECG and eyetracking data, and participants rated their aesthetic preference and state of boredom on each trial. Whereas both behavioral ratings and gaze behavior were affected by task and stimulus manipulations, EEG SNR was barely affected and generally robust across all conditions, despite only minimal preprocessing and no trial rejection. In particular, we show that using video stimuli does not necessarily result in lower EEG quality and can, on the contrary, significantly reduce eye movements while increasing both the participants' aesthetic response and general task engagement. We see these as encouraging results indicating that - at least in the lab - more liberal experimental conditions can be adopted without significant loss of signal quality.

EEG Signal Multichannel Frequency-Domain Ratio Indices for Drowsiness Detection Based on Multicriteria Optimization

Drowsiness is a risk to human lives in many occupations and activities where full awareness is essential for the safe operation of systems and vehicles, such as driving a car or flying an airplane. Although it is one of the main causes of many road accidents, there is still no reliable definition of drowsiness or a system to reliably detect it. Many researchers have observed correlations between frequency-domain features of the EEG signal and drowsiness, such as an increase in the spectral power of the theta band or a decrease in the spectral power of the beta band. In addition, features calculated as ratio indices between these frequency-domain features show further improvements in detecting drowsiness compared to frequency-domain features alone. This work aims to develop novel multichannel ratio indices that take advantage of the diversity of frequency-domain features from different brain regions. In contrast to the state-of-the-art, we use an evolutionary metaheuristic algorithm to find the nearly optimal set of features and channels from which the indices are calculated. Our results show that drowsiness is best described by the powers in delta and alpha bands. Compared to seven existing single-channel ratio indices, our two novel six-channel indices show improvements in (1) statistically significant differences observed between wakefulness and drowsiness segments, (2) precision of drowsiness detection and classification accuracy of the XGBoost algorithm and (3) model performance by saving time and memory during classification. Our work suggests that a more precise definition of drowsiness is needed, and that accurate early detection of drowsiness should be based on multichannel frequency-domain features.

The Effects of Driving Experience on the P300 Event-Related Potential during the Perception of Traffic Scenes

The incidence of human-error-related traffic collisions is markedly reduced among drivers who have few years of driving experience compared with those with little driving experience or fewer driving opportunities, even if they have a driver's license. This study analyzes the effect of driving experience on the perception of the traffic scenes through electroencephalograms (EEGs). Primarily, we focused on visual attention during driving, the essential visual function in the visual search and human gaze, and evaluated the P300, which is involved in attention, to explore the effect of driving experience on the visual attention of traffic scenes, not for improving visual ability. In the results, the P300 response was observed in both experienced and beginner drivers when they paid visual attention to the visual target. Furthermore, the latency for the peak amplitude of the P300 response among experienced drivers was markedly faster than that in beginner drivers, suggesting that the P300 latency is a piece of crucial information for driving experience on visual attention.

Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions

Fatigue poses an important safety risk to civil and military aviation. In addition to decreasing performance in-flight (chronic) fatigue has negative long-term health effects. Possible causes of fatigue include sleep loss, extended time awake, circadian phase irregularities and work load. Despite regulations limiting flight time and enabling optimal rostering, fatigue cannot be prevented completely. Especially in military operations, where limits may be extended due to operational necessities, it is impossible to rely solely on regulations to prevent fatigue. Fatigue management, consisting of preventive strategies and operational countermeasures, such as pre-flight naps and pharmaceuticals that either promote adequate sleep (hypnotics or chronobiotics) or enhance performance (stimulants), may be required to mitigate fatigue in challenging (military) aviation operations. This review describes the pathophysiology, epidemiology and effects of fatigue and its impact on aviation, as well as several aspects of fatigue management and recommendations for future research in this field.

Multimodal Assessment of the Pulse Rate Variability Analysis Module of a Photoplethysmography-Based Telemedicine System

Alterations of heart rate variability (HRV) are associated with various (patho)physiological conditions; therefore, HRV analysis has the potential to become a useful diagnostic module of wearable/telemedical devices to support remote cardiovascular/autonomic monitoring. Continuous pulse recordings obtained by photoplethysmography (PPG) can yield pulse rate variability (PRV) indices similar to HRV parameters; however, it is debated whether PRV/HRV parameters are interchangeable. In this study, we assessed the PRV analysis module of a digital arterial PPG-based telemedical system (SCN4ALL). We used Bland–Altman analysis to validate the SCN4ALL PRV algorithm to Kubios Premium software and to determine the agreements between PRV/HRV results calculated from 2-min long PPG and ECG captures recorded simultaneously in healthy individuals (n = 33) at rest and during the cold pressor test, and in diabetic patients (n = 12) at rest. We found an ideal agreement between SCN4ALL and Kubios outputs (bias < 2%). PRV and HRV parameters showed good agreements for interbeat intervals, SDNN, and RMSSD time-domain variables, for total spectral and low-frequency power (LF) frequency-domain variables, and for non-linear parameters in healthy subjects at rest and during cold pressor challenge. In diabetics, good agreements were observed for SDNN, LF, and SD2; and moderate agreement was observed for total power. In conclusion, the SCN4ALL PRV analysis module is a good alternative for HRV analysis for numerous conventional HRV parameters.

Deep Convolutional Neural Network Regularization for Alcoholism Detection Using EEG Signals

Alcoholism is attributed to regular or excessive drinking of alcohol and leads to the disturbance of the neuronal system in the human brain. This results in certain malfunctioning of neurons that can be detected by an electroencephalogram (EEG) using several electrodes on a human skull at appropriate positions. It is of great interest to be able to classify an EEG activity as that of a normal person or an alcoholic person using data from the minimum possible electrodes (or channels). Due to the complex nature of EEG signals, accurate classification of alcoholism using only a small dataset is a challenging task. Artificial neural networks, specifically convolutional neural networks (CNNs), provide efficient and accurate results in various pattern-based classification problems. In this work, we apply CNN on raw EEG data and demonstrate how we achieved 98% average accuracy by optimizing a baseline CNN model and outperforming its results in a range of performance evaluation metrics on the University of California at Irvine Machine Learning (UCI-ML) EEG dataset. This article explains the stepwise improvement of the baseline model using the dropout, batch normalization, and kernel regularization techniques and provides a comparison of the two models that can be beneficial for aspiring practitioners who aim to develop similar classification models in CNN. A performance comparison is also provided with other approaches using the same dataset.