1
|
Gao D, Wang K, Wang M, Zhou J, Zhang Y. SFT-Net: A Network for Detecting Fatigue From EEG Signals by Combining 4D Feature Flow and Attention Mechanism. IEEE J Biomed Health Inform 2024; 28:4444-4455. [PMID: 37310832 DOI: 10.1109/jbhi.2023.3285268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Fatigued driving is a leading cause of traffic accidents, and accurately predicting driver fatigue can significantly reduce their occurrence. However, modern fatigue detection models based on neural networks often face challenges such as poor interpretability and insufficient input feature dimensions. This article proposes a novel Spatial-Frequency-Temporal Network (SFT-Net) method for detecting driver fatigue using electroencephalogram (EEG) data. Our approach integrates EEG signals' spatial, frequency, and temporal information to improve recognition performance. We transform the differential entropy of five frequency bands of EEG signals into a 4D feature tensor to preserve these three types of information. An attention module is then used to recalibrate the spatial and frequency information of each input 4D feature tensor time slice. The output of this module is fed into a depthwise separable convolution (DSC) module, which extracts spatial and frequency features after attention fusion. Finally, long short-term memory (LSTM) is used to extract the temporal dependence of the sequence, and the final features are output through a linear layer. We validate the effectiveness of our model on the SEED-VIG dataset, and experimental results demonstrate that SFT-Net outperforms other popular models for EEG fatigue detection. Interpretability analysis supports the claim that our model has a certain level of interpretability. Our work addresses the challenge of detecting driver fatigue from EEG data and highlights the importance of integrating spatial, frequency, and temporal information.
Collapse
|
2
|
Hussein RM, Miften FS, George LE. Driver drowsiness detection methods using EEG signals: a systematic review. Comput Methods Biomech Biomed Engin 2023; 26:1237-1249. [PMID: 35983784 DOI: 10.1080/10255842.2022.2112574] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 07/21/2022] [Accepted: 08/08/2022] [Indexed: 11/03/2022]
Abstract
Electroencephalography (EEG) is a complex signal that may require several years of training, advanced signal processing, and feature extraction methodologies to interpret correctly. Recently, many methods have been used to extract and classify EEG data. This study reviews 62 papers that used EEG signals to detect driver drowsiness, published between January 2018 and 2022. We extract trends and highlight interesting approaches from this large body of literature to inform future research and formulate recommendations. To find relevant papers published in scientific journals, conferences, and electronic preprint repositories, researchers searched major databases covering the domains of science and engineering. For each investigation, many data items about (1) the data, (2) the channels used, (3) the extraction and classification procedure, and (4) the outcomes were extracted. These items were then analyzed one by one to uncover trends. Our analysis reveals that the amount of EEG data used across studies varies. We saw that more than half the studies used simulation driving experimental. About 21% of the studies used support vector machine (SVM), while 19% used convolutional neural networks (CNN). Overall, we can conclude that drowsiness and fatigue impair driving performance, resulting in drivers who are more exposed to risky situations.
Collapse
Affiliation(s)
- Raed Mohammed Hussein
- Iraqi Commission for Computers and Informatics, Informatics Institute of Postgraduate Studies, Baghdad, Iraq
| | - Firas Sabar Miften
- College of Education for Pure Science, University of Thi-Qar, Nasiriyah, Iraq
| | - Loay E George
- University of Information Technology & Communication, Baghdad, Iraq
| |
Collapse
|
3
|
Tan X, Wang D, Chen J, Xu M. Transformer-Based Network with Optimization for Cross-Subject Motor Imagery Identification. Bioengineering (Basel) 2023; 10:bioengineering10050609. [PMID: 37237679 DOI: 10.3390/bioengineering10050609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 05/14/2023] [Accepted: 05/16/2023] [Indexed: 05/28/2023] Open
Abstract
Exploring the effective signal features of electroencephalogram (EEG) signals is an important issue in the research of brain-computer interface (BCI), and the results can reveal the motor intentions that trigger electrical changes in the brain, which has broad research prospects for feature extraction from EEG data. In contrast to previous EEG decoding methods that are based solely on a convolutional neural network, the traditional convolutional classification algorithm is optimized by combining a transformer mechanism with a constructed end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. The use of a self-attention mechanism is studied to expand the receptive field of EEG signals to global dependence and train the neural network by optimizing the global parameters in the model. The proposed model is evaluated on a real-world public dataset and achieves the highest average accuracy of 63.56% in cross-subject experiments, which is significantly higher than that found for recently published algorithms. Additionally, good performance is achieved in decoding motor intentions. The experimental results show that the proposed classification framework promotes the global connection and optimization of EEG signals, which can be further applied to other BCI tasks.
Collapse
Affiliation(s)
- Xiyue Tan
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Dan Wang
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Jiaming Chen
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| | - Meng Xu
- Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
| |
Collapse
|
4
|
Chen XX, Ji ZG, Wang Y, Xu J, Wang LY, Wang HB. Bibliometric analysis of the effects of mental fatigue on athletic performance from 2001 to 2021. Front Psychol 2023; 13:1019417. [PMID: 36698588 PMCID: PMC9869051 DOI: 10.3389/fpsyg.2022.1019417] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Accepted: 12/09/2022] [Indexed: 01/12/2023] Open
Abstract
Aims To explore the research hot topics and main contents in the field of the influence of mental fatigue on athletic performance, and to provide new ideas and directions for future research in this field. Methods Using CiteSpace and VOSviewer visualization tool software core collection of Web of Science database to TS = ("mental fatigue" OR "mental exertion" OR "cognitive fatigue" OR "Cognitive exertion" OR "mental exhaustion" OR "mental tiredness") AND ("athletic performance" OR "technical skill*" OR "Skill*" OR "technique" OR "decision making" OR "performance") AND ("Humans") searched for the influence of mental fatigue on athletic performance from 2001 to 2021 to conduct visual analysis. Research hot topics were analyzed from the aspects of high-impact countries/regions, institutions, authors, high-frequency keywords, and mutation terms. Results A total of 658 publications were identified finally, and there has been an increasing trend in the annual number of publications, with the United States ranking first in the number of publications and influence. Future research will focus on promoting the application of EEG technology as an objective indicator for assessing mental fatigue, exploring effective methods and measures for pharmacological or non-pharmacological interventions against fatigue, and focusing on the effects of mental fatigue on endurance performance, technical skills, and sports-related decision-making. Conclusion The results of the present study help us understand the status of the mental fatigue and athletic performance field and its recent developments.
Collapse
Affiliation(s)
- Xiao-Xin Chen
- School of Psychology, Shanghai University of Sport, Shanghai, China
| | - Zhi-Guang Ji
- Department of Physical Education, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Yi Wang
- College of Rehabilitation Science, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Jing Xu
- College of Rehabilitation Science, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Li-Yan Wang
- College of Rehabilitation Science, Shanghai University of Medicine and Health Sciences, Shanghai, China
| | - Hong-Biao Wang
- Department of Physical Education, Shanghai University of Medicine and Health Sciences, Shanghai, China,*Correspondence: Hong-Biao Wang,
| |
Collapse
|
5
|
Wu D, Jiang X, Peng R. Transfer learning for motor imagery based brain-computer interfaces: A tutorial. Neural Netw 2022; 153:235-253. [PMID: 35753202 DOI: 10.1016/j.neunet.2022.06.008] [Citation(s) in RCA: 20] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Revised: 04/22/2022] [Accepted: 06/06/2022] [Indexed: 11/16/2022]
Abstract
A brain-computer interface (BCI) enables a user to communicate directly with an external device, e.g., a computer, using brain signals. It can be used to research, map, assist, augment, or repair human cognitive or sensory-motor functions. A closed-loop BCI system performs signal acquisition, temporal filtering, spatial filtering, feature engineering and classification, before sending out the control signal to an external device. Transfer learning (TL) has been widely used in motor imagery (MI) based BCIs to reduce the calibration effort for a new subject, greatly increasing their utility. This tutorial describes how TL can be considered in as many components of a BCI system as possible, and introduces a complete TL pipeline for MI-based BCIs. Examples on two MI datasets demonstrated the advantages of considering TL in multiple components of MI-based BCIs. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.
Collapse
Affiliation(s)
- Dongrui Wu
- Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Xue Jiang
- Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Ruimin Peng
- Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China.
| |
Collapse
|
6
|
Chang YC, Wang YK, Pal NR, Lin CT. Exploring Covert States of Brain Dynamics via Fuzzy Inference Encoding. IEEE Trans Neural Syst Rehabil Eng 2021; 29:2464-2473. [PMID: 34748496 DOI: 10.1109/tnsre.2021.3126264] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Human brain inherently exhibits latent mental processes which are likely to change rapidly over time. A framework that adopts a fuzzy inference system is proposed to model the dynamics of the human brain. The fuzzy inference system is used to encode real-world data to represent the salient features of the EEG signals. Then, an unsupervised clustering is conducted on the extracted feature space to identify the brain (external and covert) states that respond to different cognitive demands. To understand the human state change, a state transition diagram is introduced, allowing visualization of connectivity patterns between every pair of states. We compute the transition probability between every pair of states to represent the relationships between the states. This state transition diagram is named as the Fuzzy Covert State Transition Diagram (FCOSTD), which helps the understanding of human states and human performance. We then apply FCOSTD on distracted driving experiments. FCOSTD successfully discovers the external and covert states, faithfully reveals the transition of the brain between states, and the route of the state change when humans are distracted during a driving task. The experimental results demonstrate that different subjects have similar states and inter-state transition behaviour (establishing the consistency of the system) but different ways to allocate brain resources as different actions are being taken.
Collapse
|
7
|
Extended-Range Prediction Model Using NSGA-III Optimized RNN-GRU-LSTM for Driver Stress and Drowsiness. SENSORS 2021; 21:s21196412. [PMID: 34640732 PMCID: PMC8512694 DOI: 10.3390/s21196412] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 09/13/2021] [Accepted: 09/20/2021] [Indexed: 11/18/2022]
Abstract
Road traffic accidents have been listed in the top 10 global causes of death for many decades. Traditional measures such as education and legislation have contributed to limited improvements in terms of reducing accidents due to people driving in undesirable statuses, such as when suffering from stress or drowsiness. Attention is drawn to predicting drivers’ future status so that precautions can be taken in advance as effective preventative measures. Common prediction algorithms include recurrent neural networks (RNNs), gated recurrent units (GRUs), and long short-term memory (LSTM) networks. To benefit from the advantages of each algorithm, nondominated sorting genetic algorithm-III (NSGA-III) can be applied to merge the three algorithms. This is named NSGA-III-optimized RNN-GRU-LSTM. An analysis can be made to compare the proposed prediction algorithm with the individual RNN, GRU, and LSTM algorithms. Our proposed model improves the overall accuracy by 11.2–13.6% and 10.2–12.2% in driver stress prediction and driver drowsiness prediction, respectively. Likewise, it improves the overall accuracy by 6.9–12.7% and 6.9–8.9%, respectively, compared with boosting learning with multiple RNNs, multiple GRUs, and multiple LSTMs algorithms. Compared with existing works, this proposal offers to enhance performance by taking some key factors into account—namely, using a real-world driving dataset, a greater sample size, hybrid algorithms, and cross-validation. Future research directions have been suggested for further exploration and performance enhancement.
Collapse
|
8
|
Yang Y, Gao Z, Li Y, Wang H. A CNN identified by reinforcement learning-based optimization framework for EEG-based state evaluation. J Neural Eng 2021; 18. [PMID: 33882477 DOI: 10.1088/1741-2552/abfa71] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2020] [Accepted: 04/21/2021] [Indexed: 01/25/2023]
Abstract
Objective.Electroencephalogram (EEG) data, as a kind of complex time-series, is one of the most widely-used information measurements for evaluating human psychophysiological states. Recently, numerous works applied deep learning techniques, especially the convolutional neural network (CNN), into EEG-based research. The design of the hyper-parameters of the CNN model has a great influence on the performance of the model. Therefore, automatically designing these hyper-parameters can save the time and labor of experts. This leads to the appearance of the neural architecture search technique. In this paper, we propose a reinforcement learning (RL)-based step-by-step framework to efficiently search for CNN models.Approach.Specifically, the deep Q network in RL is first used to determine the depth of convolutional layers and the connection modes among layers. Then particle swarm optimization algorithm is used to fine-tune the number and size of convolution kernels. Through this step-by-step strategy, the search space can be narrowed in each step for saving the overall time cost. This framework is employed for both EEG-based sleep stage classification and driver drowsiness evaluation tasks.Main results.The results show that compared with state-of-the-art methods, the high-performance CNN models identified by the proposed optimization framework, can achieve high overall accuracy and better root mean squared error in the two tasks.Significance.Therefore, the proposed optimization framework has a great potential to provide high-performance results for other kinds of classification and prediction tasks. In this way, it can greatly save researchers' time cost and promote broader applications of CNNs.
Collapse
Affiliation(s)
- Yuxuan Yang
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Zhongke Gao
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Yanli Li
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - He Wang
- School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|