Convolutional Neural Network for Drowsiness Detection Using EEG Signals

A scoping review on the use of consumer-grade EEG devices for research

BACKGROUND

Commercial electroencephalography (EEG) devices have become increasingly available over the last decade. These devices have been used in a wide variety of fields ranging from engineering to cognitive neuroscience.

PURPOSE

The aim of this study was to chart peer-review articles that used consumer-grade EEG devices to collect neural data. We provide an overview of the research conducted with these relatively more affordable and user-friendly devices. We also inform future research by exploring the current and potential scope of consumer-grade EEG.

METHODS

We followed a five-stage methodological framework for a scoping review that included a systematic search using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines. We searched the following online databases: PsycINFO, MEDLINE, Embase, Web of Science, and IEEE Xplore. We charted study data according to application (BCI, experimental research, validation, signal processing, and clinical) and location of use as indexed by the first author's country.

RESULTS

We identified 916 studies that used data recorded with consumer-grade EEG: 531 were reported in journal articles and 385 in conference papers. Emotiv devices were used most, followed by the NeuroSky MindWave, OpenBCI, interaXon Muse, and MyndPlay Mindband. The most common usage was for brain-computer interfaces, followed by experimental research, signal processing, validation, and clinical purposes.

CONCLUSIONS

Consumer-grade EEG is a useful tool for neuroscientific research and will likely continue to be used well into the future. Our study provides a comprehensive review of their application, as well as future directions for researchers who plan to use these devices.

An Efficient Group Federated Learning Framework for Large-Scale EEG-Based Driver Drowsiness Detection

To avoid traffic accidents, monitoring the driver's electroencephalogram (EEG) signals to assess drowsiness is an effective solution. However, aggregating the personal data of these drivers may lead to insufficient data usage and pose a risk of privacy breaches. To address these issues, a framework called Group Federated Learning (Group-FL) for large-scale driver drowsiness detection is proposed, which can efficiently utilize diverse client data while protecting privacy. First, by arranging the clients into different levels of groups and gradually aggregating their model parameters from low-level groups to high-level groups, communication and time costs are reduced. In addition, to solve the problem of notable variations in EEG signals among different clients, a global-personalized deep neural network is designed. The global model extracts shared features from various clients, while the personalized model extracts fine-grained features from each client and outputs classification results. Finally, to address special issues such as scale/category imbalance and data pollution, three checking modules are designed for adjusting grouping, evaluating client data, and effectively applying personalized models. Through extensive experimentation, the effectiveness of each component within the framework was validated, and a mean accuracy, F1-score, and Area Under Curve (AUC) of 81.0%, 82.0%, and 87.9% was achieved, respectively, on a publicly available dataset comprising 11 subjects.

Artificial Intelligence-Powered Electronic Skin

Skin-interfaced electronics is gradually changing medical practices by enabling continuous and noninvasive tracking of physiological and biochemical information. With the rise of big data and digital medicine, next-generation electronic skin (e-skin) will be able to use artificial intelligence (AI) to optimize its design as well as uncover user-personalized health profiles. Recent multimodal e-skin platforms have already employed machine learning (ML) algorithms for autonomous data analytics. Unfortunately, there is a lack of appropriate AI protocols and guidelines for e-skin devices, resulting in overly complex models and non-reproducible conclusions for simple applications. This review aims to present AI technologies in e-skin hardware and assess their potential for new inspired integrated platform solutions. We outline recent breakthroughs in AI strategies and their applications in engineering e-skins as well as understanding health information collected by e-skins, highlighting the transformative deployment of AI in robotics, prosthetics, virtual reality, and personalized healthcare. We also discuss the challenges and prospects of AI-powered e-skins as well as predictions for the future trajectory of smart e-skins.

Driver drowsiness detection methods using EEG signals: a systematic review

Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.

A Deep Learning Approach to Classify Sitting and Sleep History from Raw Accelerometry Data during Simulated Driving

Prolonged sitting and inadequate sleep can impact driving performance. Therefore, objective knowledge of a driver's recent sitting and sleep history could help reduce safety risks. This study aimed to apply deep learning to raw accelerometry data collected during a simulated driving task to classify recent sitting and sleep history. Participants (n = 84, Mean ± SD age = 23.5 ± 4.8, 49% Female) completed a seven-day laboratory study. Raw accelerometry data were collected from a thigh-worn accelerometer during a 20-min simulated drive (8:10 h and 17:30 h each day). Two convolutional neural networks (CNNs; ResNet-18 and DixonNet) were trained to classify accelerometry data into four classes (sitting or breaking up sitting and 9-h or 5-h sleep). Accuracy was determined using five-fold cross-validation. ResNet-18 produced higher accuracy scores: 88.6 ± 1.3% for activity (compared to 77.2 ± 2.6% from DixonNet) and 88.6 ± 1.1% for sleep history (compared to 75.2 ± 2.6% from DixonNet). Class activation mapping revealed distinct patterns of movement and postural changes between classes. Findings demonstrate the suitability of CNNs in classifying sitting and sleep history using thigh-worn accelerometer data collected during a simulated drive. This approach has implications for the identification of drivers at risk of fatigue-related impairment.

Non-smooth Bayesian learning for artificial neural networks

Artificial neural networks (ANNs) are being widely used in supervised machine learning to analyze signals or images for many applications. Using an annotated learning database, one of the main challenges is to optimize the network weights. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively such as gradient-based method, Newton-type method, meta-heuristic method. For the sake of efficiency, regularization is generally used. When non-smooth regularizers are used especially to promote sparse networks, such as the ℓ 1 norm, this optimization becomes challenging due to non-differentiability issues of the target criterion. In this paper, we propose an MCMC-based optimization scheme formulated in a Bayesian framework. The proposed scheme solves the above-mentioned sparse optimization problem using an efficient sampling scheme and Hamiltonian dynamics. The designed optimizer is conducted on four (4) datasets, and the results are verified by a comparative study with two CNNs. Promising results show the usefulness of the proposed method to allow ANNs, even with low complexity levels, reaching high accuracy rates of up to 94 % . The proposed method is also faster and more robust concerning overfitting issues. More importantly, the training step of the proposed method is much faster than all competing algorithms.

Driver drowsiness estimation using EEG signals with a dynamical encoder-decoder modeling framework

Drowsiness is a leading cause of accidents on the road as it negatively affects the driver’s ability to safely operate a vehicle. Neural activity recorded by EEG electrodes is a widely used physiological correlate of driver drowsiness. This paper presents a novel dynamical modeling solution to estimate the instantaneous level of the driver drowsiness using EEG signals, where the PERcentage of eyelid CLOSure (PERCLOS) is employed as the ground truth of driver drowsiness. Applying our proposed modeling framework, we find neural features present in EEG data that encode PERCLOS. In the decoding phase, we use a Bayesian filtering solution to estimate the PERCLOS level over time. A data set that comprises 18 driving tests, conducted by 13 drivers, has been used to investigate the performance of the proposed framework. The modeling performance in estimation of PERCLOS provides robust and repeatable results in tests with manual and automated driving modes by an average RMSE of 0.117 (at a PERCLOS range of 0 to 1) and average High Probability Density percentage of 62.5%. We further hypothesized that there are biomarkers that encode the PERCLOS across different driving tests and participants. Using this solution, we identified possible biomarkers such as Theta and Delta powers. Results show that about 73% and 66% of the Theta and Delta powers which are selected as biomarkers are increasing as PERCLOS grows during the driving test. We argue that the proposed method is a robust and reliable solution to estimate drowsiness in real-time which opens the door in utilizing EEG-based measures in driver drowsiness detection systems.

Trends and Future Prospects of the Drowsiness Detection and Estimation Technology

Drowsiness is among the important factors that cause traffic accidents; therefore, a monitoring system is necessary to detect the state of a driver’s drowsiness. Driver monitoring systems usually detect three types of information: biometric information, vehicle behavior, and driver’s graphic information. This review summarizes the research and development trends of drowsiness detection systems based on various methods. Drowsiness detection methods based on the three types of information are discussed. A prospect for arousal level detection and estimation technology for autonomous driving is also presented. In the case of autonomous driving levels 4 and 5, where the driver is not the primary driving agent, the technology will not be used to detect and estimate wakefulness for accident prevention; rather, it can be used to ensure that the driver has enough sleep to arrive comfortably at the destination.

Efficient Gradient Updating Strategies with Adaptive Power Allocation for Federated Learning over Wireless Backhaul

In this paper, efficient gradient updating strategies are developed for the federated learning when distributed clients are connected to the server via a wireless backhaul link. Specifically, a common convolutional neural network (CNN) module is shared for all the distributed clients and it is trained through the federated learning over wireless backhaul connected to the main server. However, during the training phase, local gradients need to be transferred from multiple clients to the server over wireless backhaul link and can be distorted due to wireless channel fading. To overcome it, an efficient gradient updating method is proposed, in which the gradients are combined such that the effective SNR is maximized at the server. In addition, when the backhaul links for all clients have small channel gain simultaneously, the server may have severely distorted gradient vectors. Accordingly, we also propose a binary gradient updating strategy based on thresholding in which the round associated with all channels having small channel gains is excluded from federated learning. Because each client has limited transmission power, it is effective to allocate more power on the channel slots carrying specific important information, rather than allocating power equally to all channel resources (equivalently, slots). Accordingly, we also propose an adaptive power allocation method, in which each client allocates its transmit power proportionally to the magnitude of the gradient information. This is because, when training a deep learning model, the gradient elements with large values imply the large change of weight to decrease the loss function.

Deep Convolutional Neural Network Regularization for Alcoholism Detection Using EEG Signals

Alcoholism is attributed to regular or excessive drinking of alcohol and leads to the disturbance of the neuronal system in the human brain. This results in certain malfunctioning of neurons that can be detected by an electroencephalogram (EEG) using several electrodes on a human skull at appropriate positions. It is of great interest to be able to classify an EEG activity as that of a normal person or an alcoholic person using data from the minimum possible electrodes (or channels). Due to the complex nature of EEG signals, accurate classification of alcoholism using only a small dataset is a challenging task. Artificial neural networks, specifically convolutional neural networks (CNNs), provide efficient and accurate results in various pattern-based classification problems. In this work, we apply CNN on raw EEG data and demonstrate how we achieved 98% average accuracy by optimizing a baseline CNN model and outperforming its results in a range of performance evaluation metrics on the University of California at Irvine Machine Learning (UCI-ML) EEG dataset. This article explains the stepwise improvement of the baseline model using the dropout, batch normalization, and kernel regularization techniques and provides a comparison of the two models that can be beneficial for aspiring practitioners who aim to develop similar classification models in CNN. A performance comparison is also provided with other approaches using the same dataset.

A Review of EEG Signal Features and their Application in Driver Drowsiness Detection Systems

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.