1
|
Samadi E, Rahatabad FN, Nasrabadi AM, Dabanlou NJ. Brain analysis to approach human muscles synergy using deep learning. Cogn Neurodyn 2025; 19:44. [PMID: 39996071 PMCID: PMC11846801 DOI: 10.1007/s11571-025-10228-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2024] [Accepted: 01/31/2025] [Indexed: 02/26/2025] Open
Abstract
Brain signals and muscle movements have been analyzed using electroencephalogram (EEG) data in several studies. EEG signals contain a lot of noise, such as electromyographic (EMG) waves. Further studies have been done to improve the quality of the results, though it is thought that the combination of these two signals can lead to a significant improvement in the synergistic analysis of muscle movements and muscle connections. Using graph theory, this study examined the interaction of EMG and EEG signals during hand movement and estimated the synergy between muscle and brain signals. Mapping of the brain diagram was also developed to reconstruct the muscle signals from the muscle connections in the brain diagram. The proposed method included noise removal from EEG and EMG signals, graph feature analysis from EEG, and synergy calculation from EMG. Two methods were used to estimate synergy. In the first method, after calculating the brain connections, the features of the communication graph were extracted and then synergy estimating was made with neural networks. In the second method, a convolutional network created a transition from the matrix of brain connections to the synergistic EMG signal. This study reached the high correlation values of 99.8% and maximum MSE error of 0.0084. Compared to other graph-based methods, this method based on regression analysis had a very significant performance. This research can lead to the improvement of rehabilitation methods and brain-computer interfaces.
Collapse
Affiliation(s)
- Elham Samadi
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | | | | | - Nader Jafarnia Dabanlou
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| |
Collapse
|
2
|
Kang J, Lv S, Li Y, Hao P, Li X, Gao C. The effects of neurofeedback training on behavior and brain functional networks in children with autism spectrum disorder. Behav Brain Res 2025; 481:115425. [PMID: 39788456 DOI: 10.1016/j.bbr.2025.115425] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2024] [Revised: 12/21/2024] [Accepted: 01/07/2025] [Indexed: 01/12/2025]
Abstract
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder with an unclear pathogenesis to date. Neurofeedback (NFB) had shown therapeutic effects in patients with ASD. In this study,we analyzed the brain functional networks of children with ASD and investigated the impact of NFB targeting the beta rhythm training on these networks. The Autism Behavior Checklist (ABC) and Social Response Scale (SRS) were employed to evaluate the effects of NFB training on the behavioral abilities of children with ASD. We compared the differences in static and dynamic brain functional networks between ASD and Typically Developing (TD) children, also explored the changes in these networks in ASD children after 20 sessions of NFB training. The Weighted Phase Lag Index (wPLI) was used to construct static functional networks, and the Fuzzy Entropy (FuzzyEn) algorithm was further employed to measure the complexity of static functional connectivity and construct dynamic functional networks. This allowed the analysis of functional connectivity and fluctuations in the static functional networks of ASD and TD children, as well as the time variability of the dynamic functional networks. Additionally, the study explored the changes in brain functional networks and behavioral scales before and after NFB training. Results from behavioral scales indicated significant improvements in cognitive, communication, language, and social scores in ASD children following NFB intervention. EEG analysis revealed that static functional connectivity was lower, connectivity variability was higher, and temporal variability was greater in ASD children compared to TD children. Following NFB training, increased functional connectivity, reduced connectivity variability in the Delta frequency band, and decreased temporal variability were observed in ASD children. The results revealed abnormalities in both static and dynamic functional networks in children with ASD, with NFB training showed potential to modulate these networks. While our results showed that NFB training can assist participants in regulating connectivity and temporal variability in specific brain regions, robust evidence for its effectiveness in alleviating core symptoms of ASD remained limited.
Collapse
Affiliation(s)
- Jiannan Kang
- College of Electronic & Information Engineering, Hebei University, Baoding, China
| | - Shuaikang Lv
- College of Electronic & Information Engineering, Hebei University, Baoding, China
| | - Yuqi Li
- College of Electronic & Information Engineering, Hebei University, Baoding, China
| | - Pengfei Hao
- College of Electronic & Information Engineering, Hebei University, Baoding, China
| | - Xiaoli Li
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Chunxia Gao
- College of Electronic & Information Engineering, Hebei University, Baoding, China.
| |
Collapse
|
3
|
Han JC, Zhang C, Cai YD, Li YT, Shang YX, Chen ZH, Yang G, Song JJ, Su D, Bai K, Sun JT, Liu Y, Liu N, Duan Y, Wang W. Neuroimaging features for cognitive fatigue and its recovery with VR intervention: An EEG microstates analysis. Brain Res Bull 2025; 221:111223. [PMID: 39864596 DOI: 10.1016/j.brainresbull.2025.111223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2024] [Revised: 01/15/2025] [Accepted: 01/20/2025] [Indexed: 01/28/2025]
Abstract
INTRODUCTION Cognitive fatigue is mainly caused by enduring mental stress or monotonous work, impairing cognitive and physical performance. Natural scene exposure is a promising intervention for relieving cognitive fatigue, but the efficacy of virtual reality (VR) simulated natural scene exposure is unclear. We aimed to investigate the effect of VR natural scene on cognitive fatigue and further explored its underlying neurophysiological alterations with electroencephalogram (EEG) microstates analysis. METHODS Ten participants performed a 20-minute 1-back task before and after VR intervention while EEG was recorded (pre-task, post-task). Performance was measured with mean accuracy rate (MAR) and mean reaction time (MRT) of the continuous 1-back task. VR simulation of the Canal Town scene was utilized to alleviate cognitive fatigue caused by 1-back tasks. Four resting-state phases were identified: beginning, pre, post, and end phases. RESULTS Post-task had a higher MAR and a lower MRT than pre-task. For pre-task, MAR was negatively correlated with trials, while MRT was positively correlated with trials. Four EEG microstates classes (A-D) were identified, and their temporal parameters (mean duration, time coverage and occurrence) and transition probabilities were calculated. After intervention, mean duration and time coverage of class B decreased, all parameters of class C increased, while all parameters of class D decreased. Transition probabilities between classes B and D decreased but increased between classes A and C. CONCLUSION VR simulation of Canal Town scene is a potentially effective method to alleviate cognitive fatigue. Microstate is an electrophysiological trait characteristic of cognitive fatigue and might be used to indicate the effect of VR intervention.
Collapse
Affiliation(s)
- Jia-Cheng Han
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Chi Zhang
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Yan-Dong Cai
- School of Aerospace Engineering, Tsinghua University, Beijing 100084, China; Airborne Avionics Flight Test Institute, Chinese Flight Test Establishment, Xi'an, Shaanxi 710089, China.
| | - Yu-Ting Li
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Yu-Xuan Shang
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Zhu-Hong Chen
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Guan Yang
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Jia-Jie Song
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Dan Su
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Ke Bai
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Jing-Ting Sun
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| | - Yu Liu
- Hangzhou Qu'an Technology Co., Ltd, Hangzhou, Zhejiang 310000, China.
| | - Na Liu
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China; Department of Nursing, The Fourth Military Medical University, Xi'an, Shaanxi 710038, China.
| | - Ya Duan
- School of Aerospace Engineering, Tsinghua University, Beijing 100084, China; Airborne Avionics Flight Test Institute, Chinese Flight Test Establishment, Xi'an, Shaanxi 710089, China.
| | - Wen Wang
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, Tangdu Hospital, The Fourth Military Medical University, No. 569 Xinsi Road, Xi'an, Shaanxi 710038, China.
| |
Collapse
|
4
|
Guo H, Chen S, Zhou Y, Xu T, Zhang Y, Ding H. A hybrid critical channels and optimal feature subset selection framework for EEG fatigue recognition. Sci Rep 2025; 15:2139. [PMID: 39819993 PMCID: PMC11739579 DOI: 10.1038/s41598-025-86234-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2024] [Accepted: 01/09/2025] [Indexed: 01/19/2025] Open
Abstract
Fatigue driving is one of the potential factors threatening road safety, and monitoring drivers' mental state through electroencephalography (EEG) can effectively prevent such risks. In this paper, a new model, DE-GFRJMCMC, is proposed for selecting critical channels and optimal feature subsets from EEG data to improve the accuracy of fatigue driving recognition. The model is validated on the SEED-VIG dataset. The model first selects critical EEG channels using the Differential Evolution (DE) algorithm, extracting important electrode channel information to enhance recognition accuracy. These electrode channels are used to construct a Functional Brain Network (FBN), from which the topological feature set is extracted. Empirical Mode Decomposition (EMD) is then applied to extract the intrinsic mode components as network nodes, thereby reducing the influence of the number of electrode channels on the brain functional network. The topological features extracted from these components form the suboptimal feature set. To minimize redundant information, we propose an improved Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm for selecting the optimal feature subset, ensuring both the efficiency and accuracy of fatigue recognition. The optimal feature subsets were input into various classifiers, and the results showed that the K-Nearest Neighbor (KNN)-based classifier achieved the highest recognition accuracy of 96.11% ± 0.43%, demonstrating the method's stability and robustness. Compared to similar studies, this model shows superior performance in fatigue driving recognition, which is of significant value for research on fatigue driving detection and prevention.
Collapse
Affiliation(s)
- Hanying Guo
- School of Automobile and Transportation, Xihua University, Chengdu, Sichuan, China.
| | - Siying Chen
- School of Automobile and Transportation, Xihua University, Chengdu, Sichuan, China
| | - Yongjiang Zhou
- School of Automobile and Transportation, Xihua University, Chengdu, Sichuan, China
- School of Transportation and Logistics, Southwest Jiaotong University, Chengdu, Sichuan, China
| | - Ting Xu
- School of Automobile and Transportation, Xihua University, Chengdu, Sichuan, China
| | - Yuhao Zhang
- College of Traffic and Transportation, Chongqing Jiaotong University, Chongqing, China
| | - Hongliang Ding
- College of Smart City and Transportation, Southwest Jiaotong University, Chengdu, Sichuan, China
| |
Collapse
|
5
|
Ma Y, Huang J, Liu C, Shi M. A portable EEG signal acquisition system and a limited-electrode channel classification network for SSVEP. Front Neurorobot 2025; 18:1502560. [PMID: 39882377 PMCID: PMC11774901 DOI: 10.3389/fnbot.2024.1502560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2024] [Accepted: 12/30/2024] [Indexed: 01/31/2025] Open
Abstract
Brain-computer interfaces (BCIs) have garnered significant research attention, yet their complexity has hindered widespread adoption in daily life. Most current electroencephalography (EEG) systems rely on wet electrodes and numerous electrodes to enhance signal quality, making them impractical for everyday use. Portable and wearable devices offer a promising solution, but the limited number of electrodes in specific regions can lead to missing channels and reduced BCI performance. To overcome these challenges and enable better integration of BCI systems with external devices, this study developed an EEG signal acquisition platform (Gaitech BCI) based on the Robot Operating System (ROS) using a 10-channel dry electrode EEG device. Additionally, a multi-scale channel attention selection network based on the Squeeze-and-Excitation (SE) module (SEMSCS) is proposed to improve the classification performance of portable BCI devices with limited channels. Steady-state visual evoked potential (SSVEP) data were collected using the developed BCI system to evaluate both the system and network performance. Offline data from ten subjects were analyzed using within-subject and cross-subject experiments, along with ablation studies. The results demonstrated that the SEMSCS model achieved better classification performance than the comparative reference model, even with a limited number of channels. Additionally, the implementation of online experiments offers a rational solution for controlling external devices via BCI.
Collapse
Affiliation(s)
| | - Jinming Huang
- College of Engineering, Qufu Normal University, Rizhao, China
| | | | | |
Collapse
|
6
|
Mi P, Yan L, Cheng Y, Liu Y, Wang J, Shoukat MU, Yan F, Qin G, Han P, Zhai Y. Driver Cognitive Architecture Based on EEG Signals: A Review. IEEE SENSORS JOURNAL 2024; 24:36261-36286. [DOI: 10.1109/jsen.2024.3471699] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/12/2025]
Affiliation(s)
- Peiwen Mi
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Lirong Yan
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Yu Cheng
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Yan Liu
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Jun Wang
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | | | - Fuwu Yan
- School of Automotive Engineering, Wuhan University of Technology, Wuhan, China
| | - Guofeng Qin
- Teachers College for Vocational and Education, Guangxi Normal University, Guilin, China
| | - Peng Han
- Xiangyang DAAN Automobile Test Center Corporation Ltd., Xiangyang, China
| | - Yikang Zhai
- Xiangyang DAAN Automobile Test Center Corporation Ltd., Xiangyang, China
| |
Collapse
|
7
|
Wang F, Luo A, Chen D. Real-time EEG-based detection of driving fatigue using a novel semi-dry electrode with self-replenishment of conductive fluid. Comput Methods Biomech Biomed Engin 2024:1-18. [PMID: 39494681 DOI: 10.1080/10255842.2024.2423268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2024] [Revised: 09/23/2024] [Accepted: 10/20/2024] [Indexed: 11/05/2024]
Abstract
A novel semi-dry electrode that can realize self-replenishment of conductive liquid is proposed in this study. Driving fatigue is detected by extracting the refined composite multiscale fluctuation dispersion entropy (RCMFDE) features in electroencephalogram (EEG) signals collected by this electrode. The results show that the new semi-dry electrode can automatically complete the conductive fluid supplement according to its own humidity conditions, which not only notably improves the effective working time, but also significantly reduces the skin impedance. By comparing with the classical entropy algorithms, the computational speed and the stability of the RCMFDE method are Substantially enhanced.
Collapse
Affiliation(s)
- Fuwang Wang
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| | - Anni Luo
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| | - Daping Chen
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, China
| |
Collapse
|
8
|
Chen K, Chai S, Xie T, Liu Q, Ma L. EEG spatial inter-channel connectivity analysis: A GCN-based dual stream approach to distinguish mental fatigue status. Artif Intell Med 2024; 157:102996. [PMID: 39406075 DOI: 10.1016/j.artmed.2024.102996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Revised: 09/02/2024] [Accepted: 09/30/2024] [Indexed: 11/14/2024]
Abstract
Mental fatigue is defined as a decline in the ability and efficiency of mental activities. A lot of research suggests that the transition from alertness to fatigue is accompanied by alterations in correlation patterns among various brain regions. However, conventional methods for detecting mental fatigue seldom emphases inter-channel connectivity in the spatial domain. To fill this gap, this paper explores the spatial inter-channel connectivity in alertness and fatigue, employing spectral graph convolutional networks (GCN) for mental fatigue detection. We utilized Pearson correlation coefficients (PCC) to establish temporal connections and magnitude-squared coherence (MSC) for spectral connections. Topological features of the brain network were then analysed. To enhance the learning of spatial inter-channel connectivity, a dual-graph strategy transforms edge features into node features, serving as inputs to the spectral GCN. By simultaneously learning PCC and MSC features, the model results indicate significant differences in some brain network characteristics between alert and fatigue states. It confirms that the synchronicity of brain operations differs in the alert state compared to mental fatigue, and indicates that fatigue states can influence correlation patterns among different brain regions. Our approach is evaluated on a self-designed experimental dataset containing 7 subjects, demonstrating a classification accuracy of 89.59 % in group-level experiments and 95.24 % at the subject level. Additionally, on the public dataset SEED-VIG containing 23 subjects, our method achieves an accuracy of 86.58 %. In summary, this paper proposes a neural network approach based on a dynamic functional connectivity network. The network integrates both temporal and spectral connections with the goal of simultaneously learning spatial inter-channel connectivity in time and frequency domains. This effectively accomplishes fatigue state detection, highlighting that fatigue significantly influences correlations among different brain regions.
Collapse
Affiliation(s)
- Kun Chen
- School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China.
| | - Shulong Chai
- School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China.
| | - Tianli Xie
- School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China.
| | - Quan Liu
- School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China.
| | - Li Ma
- School of Information Engineering, Wuhan University of Technology, Wuhan, 430070, China.
| |
Collapse
|
9
|
Han JC, Bai K, Zhang C, Liu N, Yang G, Shang YX, Song JJ, Su D, Hao Y, Feng XL, Li SR, Wang W. Objective assessment of cognitive fatigue: a bibliometric analysis. Front Neurosci 2024; 18:1479793. [PMID: 39554851 PMCID: PMC11566139 DOI: 10.3389/fnins.2024.1479793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2024] [Accepted: 10/18/2024] [Indexed: 11/19/2024] Open
Abstract
Aim The objective of this study was to gain insight into the nature of cognitive fatigue and to identify future trends of objective assessment techniques in this field. Methods One thousand and eighty-five articles were retrieved from the Web of Science Core Collection database. R version 4.3.1, VOSviewer 1.6.20, CiteSpace 6.2.R4, and Microsoft Excel 2019 were used to perform the analysis. Results A total of 704 institutes from 56 countries participated in the relevant research, while the People's Republic of China contributed 126 articles and was the leading country. The most productive institute was the University of Gothenburg. Johansson Birgitta from the University of Gothenburg has posted the most articles (n = 13). The PLOS ONE published most papers (n = 38). The Neurosciences covered the most citations (n = 1,094). A total of 3,116 keywords were extracted and those with high frequency were mental fatigue, performance, quality-of-life, etc. Keywords mapping analysis indicated that cognitive fatigue caused by continuous work and traumatic brain injury, as well as its rehabilitation, have become the current research trend. The most co-cited literature was published in Sports Medicine. The strongest citation burst was related to electroencephalogram (EEG) event-related potential and spectral power analysis. Conclusion Publication information of related literature on the objective assessment of cognitive fatigue from 2007 to 2024 was summarized, including country and institute of origin, authors, and published journal, offering the current hotspots and novel directions in this field.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | - Xiu-Long Feng
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, The Second Affiliated Hospital of Air Force Medical University, Xi’an, Shaanxi, China
| | - Si-Rui Li
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, The Second Affiliated Hospital of Air Force Medical University, Xi’an, Shaanxi, China
| | - Wen Wang
- Department of Radiology, Functional and Molecular Imaging Key Lab of Shaanxi Province, The Second Affiliated Hospital of Air Force Medical University, Xi’an, Shaanxi, China
| |
Collapse
|
10
|
Lian Z, Xu T, Yuan Z, Li J, Thakor N, Wang H. Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking. IEEE J Biomed Health Inform 2024; 28:6568-6580. [PMID: 39167519 DOI: 10.1109/jbhi.2024.3446952] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/23/2024]
Abstract
EEG-based unimodal method has demonstrated significant success in the detection of driving fatigue. Nonetheless, data from a single modality might be not sufficient to optimize fatigue detection due to incomplete information. To address this limitation and enhance the performance of driving fatigue detection, a novel multimodal architecture combining hybrid electroencephalograph (EEG) and eye tracking data was proposed in this work. Specifically, the EEG and eye tracking data were separately input into encoders, generating two one-dimensional (1D) features. Subsequently, these 1D features were fed into a cross-modal predictive alignment module to improve fusion efficiency and two 1D attention modules to enhance feature representation. Furthermore, the fused features were recognized by a linear classifier. To evaluate the effectiveness of the proposed multimodal method, comprehensive validation tasks were conducted, including intra-session, cross-session, and cross-subject evaluations. In the intra-session task, the proposed architecture achieves an exceptional average accuracy of 99.93%. Moreover, in the cross-session task, our method demonstrates an average accuracy of 88.67%, surpassing the performance of EEG-only approach by 8.52%, eye tracking-only method by 5.92%, multimodal deep canonical correlation analysis (DCCA) technique by 0.42%, and multimodal deep generalized canonical correlation analysis (DGCCA) approach by 0.84%. Similarly, in the cross-subject task, the proposed approach achieves an average accuracy of 78.19%, outperforming EEG-only method by 5.87%, eye tracking-only approach by 4.21%, DCCA method by 0.55%, and DGCCA approach by 0.44%. The experimental results conclusively illustrate the superior effectiveness of the proposed method compared to both single modality approaches and canonical correlation analysis-based multimodal methods.
Collapse
|
11
|
Gu T, Yao W, Wang F, Fu R. Research on low-power driving fatigue monitoring method based on spiking neural network. Exp Brain Res 2024; 242:2457-2471. [PMID: 39177685 DOI: 10.1007/s00221-024-06911-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2024] [Accepted: 08/18/2024] [Indexed: 08/24/2024]
Abstract
Fatigue driving is one of the leading causes of traffic accidents, and the rapid and accurate detection of driver fatigue is of paramount importance for enhancing road safety. However, the application of deep learning models in fatigue driving detection has long been constrained by high computational costs and power consumption. To address this issue, this study proposes an approach that combines Self-Organizing Map (SOM) and Spiking Neural Networks (SNN) to develop a low-power model capable of accurately recognizing the driver's mental state. Initially, spatial features are extracted from electroencephalogram (EEG) signals using the SOM network. Subsequently, the extracted weight vectors are encoded and fed into the SNN for fatigue driving classification. The research results demonstrate that the proposed method effectively considers the spatiotemporal characteristics of EEG signals, achieving efficient fatigue detection. Simultaneously, this approach successfully reduces the model's power consumption. When compared to traditional artificial neural networks, our method reduces energy consumption by approximately 12.21-42.59%.
Collapse
Affiliation(s)
- Tianshu Gu
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, 132012, China
| | - Wanchao Yao
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, 132012, China
| | - Fuwang Wang
- School of Mechanic Engineering, Northeast Electric Power University, Jilin, 132012, China.
| | - Rongrong Fu
- College of Electrical Engineering, Yanshan University, Qinhuangdao, 066004, China
| |
Collapse
|
12
|
Zhao Y, Huang Y, Liu Z, Zhou Y. The architecture of functional brain network modulated by driving under train running noise exposure. PLoS One 2024; 19:e0306729. [PMID: 39146301 PMCID: PMC11326564 DOI: 10.1371/journal.pone.0306729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Accepted: 06/22/2024] [Indexed: 08/17/2024] Open
Abstract
A noisy environment can considerably impact drivers' attention and fatigue, endangering driving safety. Consequently, this study designed a simulated driving experimental scenario to analyse the effects of noise generated during urban rail transit train operation on drivers' functional brain networks. The experiment recruited 16 participants, and the simulated driving scenario was conducted at noise levels of 50, 60, 70, and 80 dB. Functional connectivity between all electrode pairs across various frequency bands was evaluated using the weighted phase lag index (WPLI), and a brain network based on this was constructed. Graph theoretic analysis employed network global efficiency, degree, and clustering coefficient as metrics. Significant increases in the WPLI values of theta and alpha frequency bands were observed in high noise environments (70 dB, 80 dB), as well as enhanced brain synchronisation. Furthermore, concerning the topological metrics of brain networks, it was observed that the global efficiency of brain networks in theta and alpha frequency ranges, as well as the node degree and clustering coefficients, experienced substantial growth in high noise environments (70 dB, 80 dB) as opposed to 50 dB and 60 dB. This finding indicates that high-noise environments impact the reorganisation of functional brain networks, leading to a preference for network structures with improved global efficiency. Such findings may improve our understanding of the neural mechanisms of driving under noise exposure, and thus potentially reduce road accidents to some extent.
Collapse
Affiliation(s)
- Yashuai Zhao
- School of Urban Rail Transportation, Shanghai University of Engineering Science, Shanghai, P.R. China
| | - Yuanchun Huang
- School of Urban Rail Transportation, Shanghai University of Engineering Science, Shanghai, P.R. China
| | - Zhigang Liu
- School of Urban Rail Transportation, Shanghai University of Engineering Science, Shanghai, P.R. China
| | - Yifan Zhou
- School of Urban Rail Transportation, Shanghai University of Engineering Science, Shanghai, P.R. China
| |
Collapse
|
13
|
Huang KC, Tseng CY, Lin CT. EEG Information Transfer Changes in Different Daily Fatigue Levels During Drowsy Driving. IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2024; 5:180-190. [PMID: 38606398 PMCID: PMC11008798 DOI: 10.1109/ojemb.2024.3367496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 01/19/2024] [Accepted: 02/11/2024] [Indexed: 04/13/2024] Open
Abstract
A significant issue for traffic safety has been drowsy driving for decades. A number of studies have investigated the effects of acute fatigue on spectral power; and recent research has revealed that drowsy driving is associated with a variety of brain connections in a specific cortico-cortical pathway. In spite of this, it is still unclear how different brain regions are connected in drowsy driving at different levels of daily fatigue. This study identified the brain connectivity-behavior relationship among three different daily fatigue levels (low-, median- and high-fatigue) with the EEG data transfer entropy. According to the results, only low- and medium-fatigue groups demonstrated an inverted U-shaped change in connectivity from high performance to poor behavioral performance. In addition, from low- to high-fatigue groups, connectivity magnitude decreased in the frontal region and increased in the occipital region. These study results suggest that brain connectivity and driving behavior would be affected by different levels of daily fatigue.
Collapse
Affiliation(s)
- Kuan-Chih Huang
- Brain Science and Technology Center, Department of Electrical and Computer EngineeringNational Yang Ming Chiao Tung UniversityHsinchu300Taiwan
| | - Chun-Ying Tseng
- Brain Science and Technology CenterNational Yang Ming Chiao Tung UniversityHsinchu300Taiwan
| | - Chin-Teng Lin
- Australian Artificial Intelligence Institute, Faculty of Engineering and ITUniversity of Technology SydneySydneyNSW2007Australia
- Brain Science and Technology Center, Department of Electrical and Computer EngineeringNational Yang Ming Chiao Tung UniversityHsinchu300Taiwan
| |
Collapse
|
14
|
Kleeva D, Ninenko I, Lebedev MA. Resting-state EEG recorded with gel-based vs. consumer dry electrodes: spectral characteristics and across-device correlations. Front Neurosci 2024; 18:1326139. [PMID: 38370431 PMCID: PMC10873917 DOI: 10.3389/fnins.2024.1326139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2023] [Accepted: 01/05/2024] [Indexed: 02/20/2024] Open
Abstract
Introduction Recordings of electroencephalographic (EEG) rhythms and their analyses have been instrumental in basic neuroscience, clinical diagnostics, and the field of brain-computer interfaces (BCIs). While in the past such measurements have been conducted mostly in laboratory settings, recent advancements in dry electrode technology pave way to a broader range of consumer and medical application because of their greater convenience compared to gel-based electrodes. Methods Here we conducted resting-state EEG recordings in two groups of healthy participants using three dry-electrode devices, the PSBD Headband, the PSBD Headphones and the Muse Headband, and one standard gel electrode-based system, the NVX. We examined signal quality for various spatial and spectral ranges which are essential for cognitive monitoring and consumer applications. Results Distinctive characteristics of signal quality were found, with the PSBD Headband showing sensitivity in low-frequency ranges and replicating the modulations of delta, theta and alpha power corresponding to the eyes-open and eyes-closed conditions, and the NVX system performing well in capturing high-frequency oscillations. The PSBD Headphones were more prone to low-frequency artifacts compared to the PSBD Headband, yet recorded modulations in the alpha power and had a strong alignment with the NVX at the higher EEG frequencies. The Muse Headband had several limitations in signal quality. Discussion We suggest that while dry-electrode technology appears to be appropriate for the EEG rhythm-based applications, the potential benefits of these technologies in terms of ease of use and accessibility should be carefully weighed against the capacity of each given system.
Collapse
Affiliation(s)
- Daria Kleeva
- Vladimir Zelman Center for Neurobiology and Brain Rehabilitation, Skolkovo Institute of Science and Technology, Moscow, Russia
- Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
- MSU Institute for Artificial Intelligence, Lomonosov Moscow State University, Moscow, Russia
| | - Ivan Ninenko
- Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Mikhail A. Lebedev
- Faculty of Mechanics and Mathematics, Lomonosov Moscow State University, Moscow, Russia
- I. M. Sechenov Institute of Evolutionary Physiology and Biochemistry, Saint Petersburg, Russia
| |
Collapse
|
15
|
Karthikeyan R, Carrizales J, Johnson C, Mehta RK. A Window Into the Tired Brain: Neurophysiological Dynamics of Visuospatial Working Memory Under Fatigue. HUMAN FACTORS 2024; 66:528-543. [PMID: 35574703 DOI: 10.1177/00187208221094900] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
OBJECTIVE We examine the spatiotemporal dynamics of neural activity and its correlates in heart rate and its variability (HR/HRV) during a fatiguing visuospatial working memory task. BACKGROUND The neural and physiological drivers of fatigue are complex, coupled, and poorly understood. Investigations that combine the fidelity of neural indices and the field-readiness of physiological measures can facilitate measurements of fatigue states in operational settings. METHOD Sixteen healthy adults, balanced by sex, completed a 60-minute fatiguing visuospatial working memory task. Changes in task performance, subjective measures of effort and fatigue, cerebral hemodynamics, and HR/HRV were analyzed. Peak brain activation, functional and effective connections within relevant brain networks were contrasted against spectral and temporal features of HR/HRV. RESULTS Task performance elicited increased neural activation in regions responsible for maintaining working memory capacity. With the onset of time-on-task effects, resource utilization was seen to increase beyond task-relevant networks. Over time, functional connections in the prefrontal cortex were seen to weaken, with changes in the causal relationships between key regions known to drive working memory. HR/HRV indices were seen to closely follow activity in the prefrontal cortex. CONCLUSION This investigation provided a window into the neurophysiological underpinnings of working memory under the time-on-task effect. HR/HRV was largely shown to mirror changes in cortical networks responsible for working memory, therefore supporting the possibility of unobtrusive state recognition under ecologically valid conditions. APPLICATIONS Findings here can inform the development of a fieldable index for cognitive fatigue.
Collapse
|
16
|
Chen C, Ji Z, Sun Y, Bezerianos A, Thakor N, Wang H. Self-Attentive Channel-Connectivity Capsule Network for EEG-Based Driving Fatigue Detection. IEEE Trans Neural Syst Rehabil Eng 2023; 31:3152-3162. [PMID: 37494165 DOI: 10.1109/tnsre.2023.3299156] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
Deep neural networks have recently been successfully extended to EEG-based driving fatigue detection. Nevertheless, most existing models fail to reveal the intrinsic inter-channel relations that are known to be beneficial for EEG-based classification. Additionally, these models require substantial data for training, which is often impractical due to the high cost of data collection. To simultaneously address these two issues, we propose a Self-Attentive Channel-Connectivity Capsule Network (SACC-CapsNet) for EEG-based driving fatigue detection in this paper. SACC-CapsNet starts with a temporal-channel attention module to investigate the critical temporal information and important channels for driving fatigue detection, refining the input EEG signals. Subsequently, the refined EEG data are transformed into a channel covariance matrix to capture the inter-channel relations, followed by selective kernel attention to extract the highly discriminative channel-connectivity features. Finally, a capsule neural network is employed to effectively learn the relationships between connectivity features, which is more suitable for limited data. To confirm the effectiveness of SACC-CapsNet, we collected 24-channel EEG data from 31 subjects (mean age=23.13±2.68 years, male/female=18/13) in a simulated fatigue driving environment. Extensive experiments were conducted with the acquired data, and the comparison results show that our proposed model outperforms state-of-the-art methods. Additionally, the channel covariance matrix learned from SACC-CapsNet reveals that the frontal pole is most informative for detecting driving fatigue, followed by the parietal and central regions. Intriguingly, the temporal-channel attention module can enhance the significance of these critical regions, and the reconstructed channel covariance matrix generated by the decoder network of SACC-CapsNet can effectively preserve valuable information about them.
Collapse
|
17
|
Hussain MI, Rafique MA, Kim J, Jeon M, Pedrycz W. Artificial Proprioceptive Reflex Warning Using EMG in Advanced Driving Assistance System. IEEE Trans Neural Syst Rehabil Eng 2023; 31:1635-1644. [PMID: 37028308 DOI: 10.1109/tnsre.2023.3254151] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2023]
Abstract
A frequent cause of auto accidents is disregarding the proximal traffic of an ego-vehicle during lane changing. Presumably, in a split-second-decision situation we may prevent an accident by predicting the intention of a driver before her action onset using the neural signals data, meanwhile building the perception of surroundings of a vehicle using optical sensors. The prediction of an intended action fused with the perception can generate an instantaneous signal that may replenish the driver's ignorance about the surroundings. This study examines electromyography (EMG) signals to predict intention of a driver along perception building stack of an autonomous driving system (ADS) in building an advanced driving assistant system (ADAS). EMG are classified into left-turn and right-turn intended actions and lanes and object detection with camera and Lidar are used to detect vehicles approaching from behind. A warning issued before the action onset, can alert a driver and may save her from a fatal accident. The use of neural signals for intended action prediction is a novel addition to camera, radar and Lidar based ADAS systems. Furthermore, the study demonstrates efficacy of the proposed idea with experiments designed to classify online and offline EMG data in real-world settings with computation time and the latency of communicated warnings.
Collapse
Affiliation(s)
- Muhammad Ishfaq Hussain
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea
| | - Muhammad Aasim Rafique
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea
| | - Joonmo Kim
- Department of Computer Engineering, Dankook University, Yongin, South Korea
| | - Moongu Jeon
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea
| | - Witold Pedrycz
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada
| |
Collapse
|
18
|
Xu T, Dragomir A, Liu X, Yin H, Wan F, Bezerianos A, Wang H. An EEG study of human trust in autonomous vehicles based on graphic theoretical analysis. Front Neuroinform 2022; 16:907942. [PMID: 36051853 PMCID: PMC9426721 DOI: 10.3389/fninf.2022.907942] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 06/16/2022] [Indexed: 11/22/2022] Open
Abstract
With the development of autonomous vehicle technology, human-centered transport research will likely shift to the interaction between humans and vehicles. This study focuses on the human trust variation in autonomous vehicles (AVs) as the technology becomes increasingly intelligent. This study uses electroencephalogram data to analyze human trust in AVs during simulated driving conditions. Two driving conditions, the semi-autonomous and the autonomous, which correspond to the two highest levels of automatic driving, are used for the simulation, accompanied by various driving and car conditions. The graph theoretical analysis (GTA) is the primary method for data analysis. In semi-autonomous driving mode, the local efficiency and cluster coefficient are lower in car-normal conditions than in car-malfunction conditions with the car approaching. This finding suggests that the human brain has a strong information processing ability while facing predictable potential hazards. However, when it comes to a traffic light with a car malfunctioning under the semi-autonomous driving mode, the characteristic path length is higher for the car malfunction manifesting a weak information processing ability while facing unpredictable potential hazards. Furthermore, in fully automatic driving conditions, participants cannot do anything and need low-level brain function to take emergency actions as lower local efficiency and small worldness for car malfunction. Our results shed light on the design of the human-machine interaction and human factor engineering on the high level of an autonomous vehicle.
Collapse
Affiliation(s)
- Tao Xu
- The Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, China
| | - Andrei Dragomir
- The N1 Institute, National University of Singapore, Singapore, Singapore
| | - Xucheng Liu
- Department of Electrical and Computer Engineering, Faculty of Science and Technology, University of Macau, Macau, Macao SAR, China
| | - Haojun Yin
- The Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, China
| | - Feng Wan
- Department of Electrical and Computer Engineering, Faculty of Science and Technology, University of Macau, Macau, Macao SAR, China
| | - Anastasios Bezerianos
- Hellenic Institute of Transport (HIT), Centre for Research and Technology Hellas (CERTH), Thessaloniki, Greece
| | - Hongtao Wang
- The Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, China
| |
Collapse
|
19
|
Effects of Rest-Break on mental fatigue recovery based on EEG dynamic functional connectivity. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
20
|
Assessment of 3D Visual Discomfort Based on Dynamic Functional Connectivity Analysis with HMM in EEG. Brain Sci 2022; 12:brainsci12070937. [PMID: 35884743 PMCID: PMC9313185 DOI: 10.3390/brainsci12070937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Revised: 07/09/2022] [Accepted: 07/11/2022] [Indexed: 11/16/2022] Open
Abstract
Stereoscopic displays can induce visual discomfort despite their wide application. Electroencephalography (EEG) technology has been applied to assess 3D visual discomfort, because it can capture brain activities with high temporal resolution. Previous studies explored the frequency and temporal features relevant to visual discomfort in EEG data. Recently, it was demonstrated that functional connectivity between brain regions fluctuates with time. However, the relationship between 3D visual discomfort and dynamic functional connectivity (DFC) remains unknown. Although HMM showed advantages over the sliding window method in capturing the temporal fluctuations of DFC at a single time point in functional magnetic resonance imaging (fMRI) data, it is unclear whether HMM works well in revealing the time-varying functional connectivity of EEG data. In this study, the hidden Markov model (HMM) was introduced to DFC analysis of EEG data for the first time and was used to investigate the DFC features that can be used to assess 3D visual discomfort. The results indicated that state 2, with strong connections between electrodes, occurred more frequently in the early period, whereas state 4, with overall weak connections between electrodes, occurred more frequently in the late period for both visual comfort and discomfort stimuli. Moreover, the 3D visual discomfort stimuli caused subjects to stay in state 4 more frequently, especially in the later period, in contrast to the 3D visual comfort stimuli. The results suggest that the increasing occurrence of state 4 was possibly related to visual discomfort and that the occurrence frequency of state 4 may be used to assess visual discomfort.
Collapse
|
21
|
NDCN-Brain: An Extensible Dynamic Functional Brain Network Model. Diagnostics (Basel) 2022; 12:diagnostics12051298. [PMID: 35626453 PMCID: PMC9142118 DOI: 10.3390/diagnostics12051298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 05/17/2022] [Accepted: 05/23/2022] [Indexed: 11/17/2022] Open
Abstract
As an extension of the static network, the dynamic functional brain network can show continuous changes in the brain’s connections. Then, limited by the length of the fMRI signal, it is difficult to show every instantaneous moment in the construction of a dynamic network and there is a lack of effective prediction of the dynamic changes of the network after the signal ends. In this paper, an extensible dynamic brain function network model is proposed. The model utilizes the ability of extracting and predicting the instantaneous state of the dynamic network of neural dynamics on complex networks (NDCN) and constructs a dynamic network model structure that can provide more than the original signal range. Experimental results show that every snapshot in the network obtained by the proposed method has a usable network structure and that it also has a good classification result in the diagnosis of cognitive impairment diseases.
Collapse
|
22
|
Gao L, Yu J, Zhu L, Wang S, Yuan J, Li G, Cai J, Qi X, Sun Y, Sun Y. Dynamic Reorganization of Functional Connectivity During Post-break Task Reengagement. IEEE Trans Neural Syst Rehabil Eng 2022; 30:157-166. [PMID: 35025746 DOI: 10.1109/tnsre.2022.3142855] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Because of the undesired fatigue-related consequences, accumulating efforts have been made to find an effective intervention to alleviate the suboptimal cognitive function caused by mental fatigue. Nonetheless, limitations of intervention and evaluation methods may hinder the revealing of underlying neural mechanisms of fatigue recovery. Through the newly-developed dynamic functional connectivity (FC) analysis framework, this study aims to investigate the effects of two types of mid-task interventions (i.e., rest-break and moderate-intensity exercise-break) on the dynamic reorganization of FC during the execution of psychomotor vigilance test (PVT). Using a sliding window approach, temporal brain networks within each frequency band (i.e., δ, θ, α, & β) were estimated before and immediate after the intervention, and towards the end of the task to investigate the immediate and delayed effects respectively during post-break task reengagement. Behaviourally, similar beneficial effects of exercise- and rest-break on performance were observed, manifested by the immediate improvements after both interventions and a long-lasting influence towards the end of tasks. Moreover, temporal brain networks assessment showed significant immediate decreases of fluctuability, which followed by an increase of fluctuability towards the end of intervention tasks. Furthermore, the temporal nodal measure revealed the channels with significant differences across tasks were mainly resided in the fronto-parietal areas that exhibited interesting frequency-dependent distribution. The observations of immediate and delayed dynamic FC reorganizations extend previous fatigue-related intervention and static FC studies, and provide new insight into the dynamic characteristics of FC during post-break task reengagement.
Collapse
|
23
|
Driving-Induced Neurological Biomarkers in an Advanced Driver-Assistance System. SENSORS 2021; 21:s21216985. [PMID: 34770304 PMCID: PMC8588463 DOI: 10.3390/s21216985] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Revised: 10/15/2021] [Accepted: 10/18/2021] [Indexed: 11/16/2022]
Abstract
Physiological signals are immediate and sensitive to neurological changes resulting from the mental workload induced by various driving environments and are considered a quantifying tool for understanding the association between neurological outcomes and driving cognitive workloads. Neurological assessment, outside of a highly-equipped clinical setting, requires an ambulatory electroencephalography (EEG) headset. This study aimed to quantify neurological biomarkers during a resting state and two different scenarios of driving states in a virtual driving environment. We investigated the neurological responses of seventeen healthy male drivers. EEG data were measured in an initial resting state, city-roadways driving state, and expressway driving state using a portable EEG headset in a driving simulator. During the experiment, the participants drove while experiencing cognitive workloads due to various driving environments, such as road traffic conditions, lane changes of surrounding vehicles, the speed limit, etc. The power of the beta and gamma bands decreased, and the power of the delta waves, theta, and frontal theta asymmetry increased in the driving state relative to the resting state. Delta-alpha ratio (DAR) and delta-theta ratio (DTR) showed a strong correlation with a resting state, city-roadways driving state, and expressway driving state. Binary machine-learning (ML) classification models showed a near-perfect accuracy between the resting state and driving state. Moderate classification performances were observed between the resting state, city-roadways state, and expressway state in multi-class classification. An EEG-based neurological state prediction approach may be utilized in an advanced driver-assistance system (ADAS).
Collapse
|
24
|
Wang R, Su X, Chang Z, Lin P, Wu Y. Flexible brain transitions between hierarchical network segregation and integration associated with cognitive performance during a multisource interference task. IEEE J Biomed Health Inform 2021; 26:1835-1846. [PMID: 34648461 DOI: 10.1109/jbhi.2021.3119940] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Cognition involves locally segregated and globally integrated processing. This process is hierarchically organized and linked to evidence from hierarchical modules in brain networks. However, researchers have not clearly determined how flexible transitions between these hierarchical processes are associated with cognitive behavior. Here, we designed a multisource interference task (MSIT) and introduced the nested-spectral partition (NSP) method to detect hierarchical modules in brain functional networks. By defining hierarchical segregation and integration across multiple levels, we showed that the MSIT requires higher network segregation in the whole brain and most functional systems but generates higher integration in the control system. Meanwhile, brain networks have more flexible transitions between segregated and integrated configurations in the task state. Crucially, higher functional flexibility in the resting state, less flexibility in the task state and more efficient switching of the brain from resting to task states were associated with better task performance. Our hierarchical modular analysis was more effective at detecting alterations in functional organization and the phenotype of cognitive performance than graph-based network measures at a single level.
Collapse
|
25
|
Fatigue driving recognition based on deep learning and graph neural network. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102598] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
26
|
Liu X, Li G, Wang S, Wan F, Sun Y, Wang H, Bezerianos A, Li C, Sun Y. Toward practical driving fatigue detection using three frontal EEG channels: a proof-of-concept study. Physiol Meas 2021; 42. [PMID: 33780920 DOI: 10.1088/1361-6579/abf336] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Accepted: 03/29/2021] [Indexed: 11/12/2022]
Abstract
Objective. Although various driving fatigue detection strategies have been introduced, the limited practicability is still an obstacle for the real application of these technologies. This study is based on the newly proposed non-hair-bearing (NHB) method to achieve practical driving fatigue detection with fewer channels from NHB areas and more efficient electroencephalogram (EEG) features.Approach. EEG data were recorded from 20 healthy subjects (15 males, age = 22.2 ± 3.2 years) in a 90 min simulated driving task using a remote wireless cap. Behaviorally, subjects demonstrated a salient fatigue effect, as reflected by a monotonic increase in reaction time. Using a sliding-window approach, we determined the vigilant and fatigued states at individual level to reduce the inter-subject differences in behavioral impairment and brain activity. Multiple EEG features, including power-spectrum density (PSD), functional connectivity (FC), and entropy, were estimated in a pairwise manner, which were set as input for fatigue classification.Main results. Intriguingly, this data-driven approach showed that the best classification performance was achieved using three EEG channel pairs located in the NHB area. The mixed features of the frontal NHB area lead to the high within-subject detection rate of driving fatigue (92.7% ± 0.92%) with satisfactory generalizability for fatigue classification across different subjects (77.13% ± 0.85%). Moreover, we found the most prominent contributing features were PSD of different frequency bands within the frontal NHB area and FC within the frontal NHB area and between frontal and parietal areas.Significance. In summary, the current work provided objective evidence to support the effectiveness of the NHB method and further improved the performance, thereby moving a step forward towards practical driving fatigue detection in real-world scenarios.
Collapse
Affiliation(s)
- Xucheng Liu
- Department of Electrical and Computer Engineering, Faculty of Science and Technology, University of Macau, Taipa, Macau.,Centre for Cognitive and Brain Sciences, Institute of Collaborative Innovation, University of Macau, Paipa, Macau
| | - Gang Li
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, Zhejiang University, Zhejiang, People's Republic of China.,College of Engineering, Zhejiang Normal University, Zhejiang, People's Republic of China
| | - Sujie Wang
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, Zhejiang University, Zhejiang, People's Republic of China
| | - Feng Wan
- Department of Electrical and Computer Engineering, Faculty of Science and Technology, University of Macau, Taipa, Macau.,Centre for Cognitive and Brain Sciences, Institute of Collaborative Innovation, University of Macau, Paipa, Macau
| | - Yi Sun
- Department of Neurology, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Zhejiang, People's Republic of China
| | - Hongtao Wang
- Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen, People's Republic of China
| | - Anastasios Bezerianos
- The N1 Institute for Health, National University of Singapore, Singapore.,Hellenic Institute of Transportation, Centre for Research and Technology Hellas, Thessaloniki, Greece
| | - Chuantao Li
- Naval Medical Center of PLA, Department of Aviation Medicine, Naval Military Medical University, Shanghai, People's Republic of China
| | - Yu Sun
- Key Laboratory for Biomedical Engineering of Ministry of Education, Department of Biomedical Engineering, Zhejiang University, Zhejiang, People's Republic of China.,Zhejiang Lab, Zhejiang, People's Republic of China
| |
Collapse
|
27
|
Wang H, Xu L, Bezerianos A, Chen C, Zhang Z. Linking Attention-Based Multiscale CNN With Dynamical GCN for Driving Fatigue Detection. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT 2021; 70:1-11. [PMID: 0 DOI: 10.1109/tim.2020.3047502] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
|