1
|
Luo Y, Liu W, Li H, Lu Y, Lu BL. A cross-scenario and cross-subject domain adaptation method for driving fatigue detection. J Neural Eng 2024; 21:046004. [PMID: 38838664 DOI: 10.1088/1741-2552/ad546d] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 06/05/2024] [Indexed: 06/07/2024]
Abstract
Objective.The scarcity of electroencephalogram (EEG) data, coupled with individual and scenario variations, leads to considerable challenges in real-world EEG-based driver fatigue detection. We propose a domain adaptation method that utilizes EEG data collected from a laboratory to supplement real-world EEG data and constructs a cross-scenario and cross-subject driver fatigue detection model for real-world scenarios.Approach.First, we collect EEG data from subjects participating in a driving experiment conducted in both laboratory and real-world scenarios. To address the issue of data scarcity, we build a real-world fatigued driving detection model by integrating the real-world data with the laboratory data. Then, we propose a method named cross-scenario and cross-subject domain adaptation (CS2DA), which aims to eliminate the domain shift problem caused by individual variances and scenario differences. Adversarial learning is adopted to extract the common features observed across different subjects within the same scenario. The multikernel maximum mean discrepancy (MK-MMD) method is applied to further minimize scenario differences. Additionally, we propose a conditional MK-MMD constraint to better utilize label information. Finally, we use seven rules to fuse the predicted labels.Main results.We evaluate the CS2DA method through extensive experiments conducted on the two EEG datasets created in this work: the SEED-VLA and the SEED-VRW datasets. Different domain adaptation methods are used to construct a real-world fatigued driving detection model using data from laboratory and real-world scenarios, as well as a combination of both. Our findings show that the proposed CS2DA method outperforms the existing traditional and adversarial learning-based domain adaptation approaches. We also find that combining data from both laboratory and real-world scenarios improves the performance of the model.Significance.This study contributes two EEG-based fatigue driving datasets and demonstrates that the proposed CS2DA method can effectively enhance the performance of a real-world fatigued driving detection model.
Collapse
Affiliation(s)
- Yun Luo
- RuiJin-Mihoyo Laboratory, Clinical Neuroscience Center, RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin 2nd Rd., Shanghai 200020, People's Republic of China
| | - Wei Liu
- RuiJin-Mihoyo Laboratory, Clinical Neuroscience Center, RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin 2nd Rd., Shanghai 200020, People's Republic of China
| | - Hanqi Li
- Disaster Information Systems with Information Technology, Disaster Prevention Research Institute, Kyoto University, Gokasho, Uji, Kyoto 611-0011, Japan
| | - Yong Lu
- RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin 2nd Rd., Shanghai 200020, People's Republic of China
| | - Bao-Liang Lu
- RuiJin-Mihoyo Laboratory, Clinical Neuroscience Center, RuiJin Hospital, Shanghai Jiao Tong University School of Medicine, 197 Ruijin 2nd Rd., Shanghai 200020, People's Republic of China
- Department of Computer Science and Engineering, Center for Brain-Like Computing and Machine Intelligence, Shanghai Jiao Tong University, 800 Dong Chuan Rd., Shanghai 200240, People's Republic of China
| |
Collapse
|
2
|
Chen X, Niu Y, Zhao Y, Qin X. An Efficient Group Federated Learning Framework for Large-Scale EEG-Based Driver Drowsiness Detection. Int J Neural Syst 2024; 34:2450003. [PMID: 37964570 DOI: 10.1142/s0129065724500035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2023]
Abstract
To avoid traffic accidents, monitoring the driver's electroencephalogram (EEG) signals to assess drowsiness is an effective solution. However, aggregating the personal data of these drivers may lead to insufficient data usage and pose a risk of privacy breaches. To address these issues, a framework called Group Federated Learning (Group-FL) for large-scale driver drowsiness detection is proposed, which can efficiently utilize diverse client data while protecting privacy. First, by arranging the clients into different levels of groups and gradually aggregating their model parameters from low-level groups to high-level groups, communication and time costs are reduced. In addition, to solve the problem of notable variations in EEG signals among different clients, a global-personalized deep neural network is designed. The global model extracts shared features from various clients, while the personalized model extracts fine-grained features from each client and outputs classification results. Finally, to address special issues such as scale/category imbalance and data pollution, three checking modules are designed for adjusting grouping, evaluating client data, and effectively applying personalized models. Through extensive experimentation, the effectiveness of each component within the framework was validated, and a mean accuracy, F1-score, and Area Under Curve (AUC) of 81.0%, 82.0%, and 87.9% was achieved, respectively, on a publicly available dataset comprising 11 subjects.
Collapse
Affiliation(s)
- Xinyuan Chen
- School of Information Science and Engineering, Shandong Normal University, Jinan 250014, P. R. China
| | - Yi Niu
- School of Information Science and Engineering, Shandong Normal University, Jinan 250014, P. R. China
| | - Yanna Zhao
- School of Information Science and Engineering, Shandong Normal University, Jinan 250014, P. R. China
| | - Xue Qin
- School of Information Science and Engineering, Shandong Normal University, Jinan 250014, P. R. China
| |
Collapse
|
3
|
Reddy YRM, Muralidhar P, Srinivas M. An Effective Hybrid Deep Learning Model for Single-Channel EEG-Based Subject-Independent Drowsiness Recognition. Brain Topogr 2024; 37:1-18. [PMID: 37995000 DOI: 10.1007/s10548-023-01016-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 10/22/2023] [Indexed: 11/24/2023]
Abstract
Nowadays, road accidents pose a severe risk in cases of sleep disorders. We proposed a novel hybrid deep-learning model for detecting drowsiness to address this issue. The proposed model combines the strengths of discrete wavelet long short-term memory (DWLSTM) and convolutional neural networks (CNN) models to classify single-channel electroencephalogram (EEG) signals. Baseline models such as support vector machine (SVM), linear discriminant analysis (LDA), back propagation neural networks (BPNN), CNN, and CNN merged with LSTM (CNN+LSTM) did not fully utilize the time sequence information. Our proposed model incorporates a majority voting between LSTM layers integrated with discrete wavelet transform (DWT) and the CNN model fed with spectrograms as images. The features extracted from sub-bands generated by DWT can provide more informative & discriminating than using the raw EEG signal. Similarly, spectrogram images fed to CNN learn the specific patterns and features with different levels of drowsiness. Furthermore, the proposed model outperformed state-of-the-art deep learning techniques and conventional baseline methods, achieving an average accuracy of 74.62%, 77.76% (using rounding, F1-score maximization approach respectively for generating labels) on 11 subjects for leave-one-out subject method. It achieved high accuracy while maintaining relatively shorter training and testing times, making it more desirable for quicker drowsiness detection. The performance metrics (accuracy, precision, recall, F1-score) are evaluated after 100 randomized tests along with a 95% confidence interval for classification. Additionally, we validated the mean accuracies from five types of wavelet families, including daubechis, symlet, bi-orthogonal, coiflets, and haar, merged with LSTM layers.
Collapse
Affiliation(s)
- Y Rama Muni Reddy
- Department of Electronics and Communication Engineering, National Institute of Technology, Warangal, Telangana, 506004, India.
| | - P Muralidhar
- Department of Electronics and Communication Engineering, National Institute of Technology, Warangal, Telangana, 506004, India
| | - M Srinivas
- Department of Computer Science Engineering, National Institute of Technology, Warangal, Telangana, 506004, India
| |
Collapse
|
4
|
Wu X, Yang J, Shao Y, Chen X. Mental fatigue assessment by an arbitrary channel EEG based on morphological features and LSTM-CNN. Comput Biol Med 2023; 167:107652. [PMID: 37950945 DOI: 10.1016/j.compbiomed.2023.107652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 10/05/2023] [Accepted: 10/31/2023] [Indexed: 11/13/2023]
Abstract
In order to achieve more sensitive mental fatigue assessment (MFA) based on an arbitrary channel EEG, this study proposed a series of feature extraction methods that combine mathematical morphology (MM), as well as an LSTM-CNN architecture. Firstly, 37 subjects had their resting-state EEGs collected at rested wakefulness (RW) and after 24 h of sleep deprivation (SD) using a 30-channel EEG acquisition device, the RW and SD groups were regarded as the negative and positive groups of mental fatigue, respectively, and the EEG collection were further categorized into two conditions: eye-opened state (EO) and eye-closed state (EC). Then, since MM can reflect the morphological characteristics of EEG rhythms and their potentials relatively independently of the time-frequency analysis and phase calculation, the MM methods were found to better reflect the mental fatigue after SD statistically, whether for single features (ANOVA: p<0.000001), multiple features (clustering by K-means, t-test: p<0.01), or time series feature spaces (calculating CD, t-test: p<0.01) of a single channel. Finally, the LSTM-CNN enhanced the generalization ability when dealing with different single-channel EEG by combining GRUs with convolutional layers: comparing the AUCs of different architectures for MFA based on an arbitrary channel, LSTM-CNN (0.992) > LSTM network (0.94) > CNN (0.831) > MLP (0.754). Moreover, the use of MM also improved the accuracy of analyzed architectures, and the true/false positive rate (TPR/FPR) of the LSTM-CNN architecture for MFA based on an arbitrary channel reached 97.024 %/3.497 %, which provided a feasible solution for the arbitrary channel EEG-based MFA.
Collapse
Affiliation(s)
- Xiaolong Wu
- School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China; Shunde Innovation School, University of Science and Technology Beijing, Guangdong, China
| | - Jianhong Yang
- School of Mechanical Engineering, University of Science and Technology Beijing, Beijing, China; Shunde Innovation School, University of Science and Technology Beijing, Guangdong, China; Technical Support Center for Prevention and Control of Disastrous Accidents in Metal Smelting, University of Science and Technology Beijing, Beijing, China.
| | - Yongcong Shao
- School of Psychology, Beijing Sport University, Beijing, China; Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, China
| | - Xuewei Chen
- Institute of Environmental and Operational Medicine, Academy of Military Sciences, Tianjin, China
| |
Collapse
|
5
|
Hussein RM, Miften FS, George LE. Driver drowsiness detection methods using EEG signals: a systematic review. Comput Methods Biomech Biomed Engin 2023; 26:1237-1249. [PMID: 35983784 DOI: 10.1080/10255842.2022.2112574] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 07/21/2022] [Accepted: 08/08/2022] [Indexed: 11/03/2022]
Abstract
Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.
Collapse
Affiliation(s)
- Raed Mohammed Hussein
- Iraqi Commission for Computers and Informatics, Informatics Institute of Postgraduate Studies, Baghdad, Iraq
| | - Firas Sabar Miften
- College of Education for Pure Science, University of Thi-Qar, Nasiriyah, Iraq
| | - Loay E George
- University of Information Technology & Communication, Baghdad, Iraq
| |
Collapse
|
6
|
Cui J, Yuan L, Wang Z, Li R, Jiang T. Towards best practice of interpreting deep learning models for EEG-based brain computer interfaces. Front Comput Neurosci 2023; 17:1232925. [PMID: 37663037 PMCID: PMC10470463 DOI: 10.3389/fncom.2023.1232925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 07/24/2023] [Indexed: 09/05/2023] Open
Abstract
Introduction As deep learning has achieved state-of-the-art performance for many tasks of EEG-based BCI, many efforts have been made in recent years trying to understand what have been learned by the models. This is commonly done by generating a heatmap indicating to which extent each pixel of the input contributes to the final classification for a trained model. Despite the wide use, it is not yet understood to which extent the obtained interpretation results can be trusted and how accurate they can reflect the model decisions. Methods We conduct studies to quantitatively evaluate seven different deep interpretation techniques across different models and datasets for EEG-based BCI. Results The results reveal the importance of selecting a proper interpretation technique as the initial step. In addition, we also find that the quality of the interpretation results is inconsistent for individual samples despite when a method with an overall good performance is used. Many factors, including model structure and dataset types, could potentially affect the quality of the interpretation results. Discussion Based on the observations, we propose a set of procedures that allow the interpretation results to be presented in an understandable and trusted way. We illustrate the usefulness of our method for EEG-based BCI with instances selected from different scenarios.
Collapse
Affiliation(s)
- Jian Cui
- Research Center for Augmented Intelligence, Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, China
| | - Liqiang Yuan
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore
| | - Zhaoxiang Wang
- Research Center for Augmented Intelligence, Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, China
| | - Ruilin Li
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore
| | - Tianzi Jiang
- Research Center for Augmented Intelligence, Research Institute of Artificial Intelligence, Zhejiang Lab, Hangzhou, China
- Brainnetome Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
7
|
Zhou X, Lin D, Jia Z, Xiao J, Liu C, Zhai L, Liu Y. An EEG Channel Selection Framework for Driver Drowsiness Detection via Interpretability Guidance. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-5. [PMID: 38083658 DOI: 10.1109/embc40787.2023.10341126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
Drowsy driving has a crucial influence on driving safety, creating an urgent demand for driver drowsiness detection. Electroencephalogram (EEG) signal can accurately reflect the mental fatigue state and thus has been widely studied in drowsiness monitoring. However, the raw EEG data is inherently noisy and redundant, which is neglected by existing works that just use single-channel EEG data or full-head channel EEG data for model training, resulting in limited performance of driver drowsiness detection. In this paper, we are the first to propose an Interpretability-guided Channel Selection (ICS) framework for the driver drowsiness detection task. Specifically, we design a two-stage training strategy to progressively select the key contributing channels with the guidance of interpretability. We first train a teacher network in the first stage using full-head channel EEG data. Then we apply the class activation mapping (CAM) to the trained teacher model to highlight the high-contributing EEG channels and further propose a channel voting scheme to select the top N contributing EEG channels. Finally, we train a student network with the selected channels of EEG data in the second stage for driver drowsiness detection. Experiments are designed on a public dataset, and the results demonstrate that our method is highly applicable and can significantly improve the performance of cross-subject driver drowsiness detection.
Collapse
|
8
|
Peng B, Zhang Y, Wang M, Chen J, Gao D. T-A-MFFNet: Multi-feature fusion network for EEG analysis and driving fatigue detection based on time domain network and attention network. Comput Biol Chem 2023; 104:107863. [PMID: 37023639 DOI: 10.1016/j.compbiolchem.2023.107863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 02/14/2023] [Accepted: 03/30/2023] [Indexed: 04/05/2023]
Abstract
Driving fatigue detection based on EEG signals is a research hotspot in applying brain-computer interfaces. EEG signal is complex, unstable, and nonlinear. Most existing methods rarely analyze the data characteristics from multiple dimensions, so it takes work to analyze the data comprehensively. To analyze EEG signals more comprehensively, this paper evaluates a feature extraction strategy of EEG data based on differential entropy (DE). This method combines the characteristics of different frequency bands, extracts the frequency domain characteristics of EEG, and retains the spatial information between channels. This paper proposes a multi-feature fusion network (T-A-MFFNet) based on the time domain and attention network. The model is composed of a time domain network (TNet), channel attention network (CANet), spatial attention network (SANet), and multi-feature fusion network(MFFNet) based on a squeeze network. T-A-MFFNet aims to learn more valuable features from the input data to achieve good classification results. Specifically, the TNet network extracts high-level time series information from EEG data. CANet and SANet are used to fuse channel and spatial features. They use MFFNet to merge multi-dimensional features and realize classification. The validity of the model is verified on the SEED-VIG dataset. The experimental results show that the accuracy of the proposed method reaches 85.65 %, which is superior to the current popular model. The proposed method can learn more valuable information from EEG signals to improve the ability to identify fatigue status and promote the development of the research field of driving fatigue detection based on EEG signals.
Collapse
|
9
|
Arif S, Munawar S, Ali H. Driving drowsiness detection using spectral signatures of EEG-based neurophysiology. Front Physiol 2023; 14:1153268. [PMID: 37064914 PMCID: PMC10097971 DOI: 10.3389/fphys.2023.1153268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Accepted: 03/09/2023] [Indexed: 03/31/2023] Open
Abstract
Introduction: Drowsy driving is a significant factor causing dire road crashes and casualties around the world. Detecting it earlier and more effectively can significantly reduce the lethal aftereffects and increase road safety. As physiological conditions originate from the human brain, so neurophysiological signatures in drowsy and alert states may be investigated for this purpose. In this preface, A passive brain-computer interface (pBCI) scheme using multichannel electroencephalography (EEG) brain signals is developed for spatially localized and accurate detection of human drowsiness during driving tasks.Methods: This pBCI modality acquired electrophysiological patterns of 12 healthy subjects from the prefrontal (PFC), frontal (FC), and occipital cortices (OC) of the brain. Neurological states are recorded using six EEG channels spread over the right and left hemispheres in the PFC, FC, and OC of the sleep-deprived subjects during simulated driving tasks. In post-hoc analysis, spectral signatures of the δ, θ, α, and β rhythms are extracted in terms of spectral band powers and their ratios with a temporal correlation over the complete span of the experiment. Minimum redundancy maximum relevance, Chi-square, and ReliefF feature selection methods are used and aggregated with a Z-score based approach for global feature ranking. The extracted drowsiness attributes are classified using decision trees, discriminant analysis, logistic regression, naïve Bayes, support vector machines, k-nearest neighbors, and ensemble classifiers. The binary classification results are reported with confusion matrix-based performance assessment metrics.Results: In inter-classifier comparison, the optimized ensemble model achieved the best results of drowsiness classification with 85.6% accuracy and precision, 89.7% recall, 87.6% F1-score, 80% specificity, 70.3% Matthews correlation coefficient, 70.2% Cohen’s kappa score, and 91% area under the receiver operating characteristic curve with 76-ms execution time. In inter-channel comparison, the best results were obtained at the F8 electrode position in the right FC of the brain. The significance of all the results was validated with a p-value of less than 0.05 using statistical hypothesis testing methods.Conclusions: The proposed scheme has achieved better results for driving drowsiness detection with the accomplishment of multiple objectives. The predictor importance approach has reduced the feature extraction cost and computational complexity is minimized with the use of conventional machine learning classifiers resulting in low-cost hardware and software requirements. The channel selection approach has spatially localized the most promising brain region for drowsiness detection with only a single EEG channel (F8) which reduces the physical intrusiveness in normal driving operation. This pBCI scheme has a good potential for practical applications requiring earlier, more accurate, and less disruptive drowsiness detection using the spectral information of EEG biosignals.
Collapse
Affiliation(s)
- Saad Arif
- Department of Mechanical Engineering, HITEC University Taxila, Taxila Cantt, Pakistan
| | - Saba Munawar
- Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah Cantt, Pakistan
| | - Hashim Ali
- Department of Computer Science, School of Engineering and Digital Sciences, Nazarbayev University, Astana, Kazakhstan
- *Correspondence: Hashim Ali,
| |
Collapse
|
10
|
Collazos-Huertas DF, Álvarez-Meza AM, Cárdenas-Peña DA, Castaño-Duque GA, Castellanos-Domínguez CG. Posthoc Interpretability of Neural Responses by Grouping Subject Motor Imagery Skills Using CNN-Based Connectivity. SENSORS (BASEL, SWITZERLAND) 2023; 23:2750. [PMID: 36904950 PMCID: PMC10007181 DOI: 10.3390/s23052750] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 02/19/2023] [Accepted: 02/27/2023] [Indexed: 06/18/2023]
Abstract
Motor Imagery (MI) refers to imagining the mental representation of motor movements without overt motor activity, enhancing physical action execution and neural plasticity with potential applications in medical and professional fields like rehabilitation and education. Currently, the most promising approach for implementing the MI paradigm is the Brain-Computer Interface (BCI), which uses Electroencephalogram (EEG) sensors to detect brain activity. However, MI-BCI control depends on a synergy between user skills and EEG signal analysis. Thus, decoding brain neural responses recorded by scalp electrodes poses still challenging due to substantial limitations, such as non-stationarity and poor spatial resolution. Also, an estimated third of people need more skills to accurately perform MI tasks, leading to underperforming MI-BCI systems. As a strategy to deal with BCI-Inefficiency, this study identifies subjects with poor motor performance at the early stages of BCI training by assessing and interpreting the neural responses elicited by MI across the evaluated subject set. Using connectivity features extracted from class activation maps, we propose a Convolutional Neural Network-based framework for learning relevant information from high-dimensional dynamical data to distinguish between MI tasks while preserving the post-hoc interpretability of neural responses. Two approaches deal with inter/intra-subject variability of MI EEG data: (a) Extracting functional connectivity from spatiotemporal class activation maps through a novel kernel-based cross-spectral distribution estimator, (b) Clustering the subjects according to their achieved classifier accuracy, aiming to find common and discriminative patterns of motor skills. According to the validation results obtained on a bi-class database, an average accuracy enhancement of 10% is achieved compared to the baseline EEGNet approach, reducing the number of "poor skill" subjects from 40% to 20%. Overall, the proposed method can be used to help explain brain neural responses even in subjects with deficient MI skills, who have neural responses with high variability and poor EEG-BCI performance.
Collapse
Affiliation(s)
| | | | | | - Germán Albeiro Castaño-Duque
- Cultura de la Calidad en la Educación Research Group, Universidad Nacional de Colombia, Manizales 170003, Colombia
| | | |
Collapse
|
11
|
Qin X, Niu Y, Zhou H, Li X, Jia W, Zheng Y. Driver Drowsiness EEG Detection Based on Tree Federated Learning and Interpretable Network. Int J Neural Syst 2023; 33:2350009. [PMID: 36655401 DOI: 10.1142/s0129065723500090] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Accurate identification of driver's drowsiness state through Electroencephalogram (EEG) signals can effectively reduce traffic accidents, but EEG signals are usually stored in various clients in the form of small samples. This study attempts to construct an efficient and accurate privacy-preserving drowsiness monitoring system, and proposes a fusion model based on tree Federated Learning (FL) and Convolutional Neural Network (CNN), which can not only identify and explain the driver's drowsiness state, but also integrate the information of different clients under the premise of privacy protection. Each client uses CNN with the Global Average Pooling (GAP) layer and shares model parameters. The tree FL transforms communication relationships into a graph structure, and model parameters are transmitted in parallel along connected branches of the graph. Moreover, the Class Activation Mapping (CAM) is used to find distinctive EEG features for representing specific classes. On EEG data of 11 subjects, it is found that this method has higher average accuracy, F1-score and AUC than the traditional classification method, reaching 73.56%, 73.26% and 78.23%, respectively. Compared with the traditional FL algorithm, this method better protects the driver's privacy and improves communication efficiency.
Collapse
Affiliation(s)
- Xue Qin
- School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, P. R. China
| | - Yi Niu
- School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, P. R. China
| | - Huiyu Zhou
- School of Computing and Mathematical Sciences, University of Leicester, Leicester, LE1 7RH, UK
| | - Xiaojie Li
- School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, P. R. China
| | - Weikuan Jia
- School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, P. R. China
| | - Yuanjie Zheng
- School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, P. R. China
| |
Collapse
|
12
|
Recognising drivers’ mental fatigue based on EEG multi-dimensional feature selection and fusion. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
13
|
Di Flumeri G, Ronca V, Giorgi A, Vozzi A, Aricò P, Sciaraffa N, Zeng H, Dai G, Kong W, Babiloni F, Borghini G. EEG-Based Index for Timely Detecting User's Drowsiness Occurrence in Automotive Applications. Front Hum Neurosci 2022; 16:866118. [PMID: 35669201 PMCID: PMC9164820 DOI: 10.3389/fnhum.2022.866118] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2022] [Accepted: 04/19/2022] [Indexed: 11/13/2022] Open
Abstract
Human errors are widely considered among the major causes of road accidents. Furthermore, it is estimated that more than 90% of vehicle crashes causing fatal and permanent injuries are directly related to mental tiredness, fatigue, and drowsiness of the drivers. In particular, driving drowsiness is recognized as a crucial aspect in the context of road safety, since drowsy drivers can suddenly lose control of the car. Moreover, the driving drowsiness episodes mostly appear suddenly without any prior behavioral evidence. The present study aimed at characterizing the onset of drowsiness in car drivers by means of a multimodal neurophysiological approach to develop a synthetic electroencephalographic (EEG)-based index, able to detect drowsy events. The study involved 19 participants in a simulated scenario structured in a sequence of driving tasks under different situations and traffic conditions. The experimental conditions were designed to induce prominent mental drowsiness in the final part. The EEG-based index, so-called “MDrow index”, was developed and validated to detect the driving drowsiness of the participants. The MDrow index was derived from the Global Field Power calculated in the Alpha EEG frequency band over the parietal brain sites. The results demonstrated the reliability of the proposed MDrow index in detecting the driving drowsiness experienced by the participants, resulting also more sensitive and timely sensible with respect to more conventional autonomic parameters, such as the EyeBlinks Rate and the Heart Rate Variability, and to subjective measurements (self-reports).
Collapse
Affiliation(s)
- Gianluca Di Flumeri
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy.,BrainSigns srl, Rome, Italy
| | - Vincenzo Ronca
- BrainSigns srl, Rome, Italy.,Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Rome, Italy
| | - Andrea Giorgi
- BrainSigns srl, Rome, Italy.,Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Rome, Italy
| | - Alessia Vozzi
- BrainSigns srl, Rome, Italy.,Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Rome, Italy
| | - Pietro Aricò
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy.,BrainSigns srl, Rome, Italy
| | | | - Hong Zeng
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Guojun Dai
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Wanzeng Kong
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Fabio Babiloni
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy.,BrainSigns srl, Rome, Italy.,School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Gianluca Borghini
- Laboratory of Industrial Neuroscience, Department of Molecular Medicine, Sapienza University of Rome, Rome, Italy.,BrainSigns srl, Rome, Italy
| |
Collapse
|
14
|
Li F, Chen CH, Lee CH, Feng S. Artificial intelligence-enabled non-intrusive vigilance assessment approach to reducing traffic controller’s human errors. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2021.108047] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
15
|
Gupta SS, Taori TJ, Ladekar MY, Manthalkar RR, Gajre SS, Joshi YV. Classification of cross task cognitive workload using deep recurrent network with modelling of temporal dynamics. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.103070] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|