1
|
Pulferer HS, Guan C, Müller-Putz GR. Investigating multilevel cognitive processing within error-free and error-prone feedback conditions in executed and observed car driving. Front Hum Neurosci 2024; 18:1383956. [PMID: 38993330 PMCID: PMC11236611 DOI: 10.3389/fnhum.2024.1383956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Accepted: 06/17/2024] [Indexed: 07/13/2024] Open
Abstract
Accident analyses repeatedly reported the considerable contribution of run-off-road incidents to fatalities in road traffic, and despite considerable advances in assistive technologies to mitigate devastating consequences, little insight into the drivers' brain response during such accident scenarios has been gained. While various literature documents neural correlates to steering motion, the driver's mental state, and the impact of distraction and fatigue on driving performance, the cortical substrate of continuous deviations of a car from the road - i.e., how the brain represents a varying discrepancy between the intended and observed car position and subsequently assigns customized levels of corrective measures - remains unclear. Furthermore, the superposition of multiple subprocesses, such as visual and erroneous feedback processing, performance monitoring, or motor control, complicates a clear interpretation of engaged brain regions within car driving tasks. In the present study, we thus attempted to disentangle these subprocesses, employing passive and active steering conditions within both error-free and error-prone vehicle operation conditions. We recorded EEG signals of 26 participants in 13 sessions, simultaneously measuring pairs of Executors (actively steering) and Observers (strictly observing) during a car driving task. We observed common brain patterns in the Executors regardless of error-free or error-prone vehicle operation, albeit with a shift in spectral activity from motor beta to occipital alpha oscillations within erroneous conditions. Further, significant frontocentral differences between Observers and Executors, tracing back to the caudal anterior cingulate cortex, arose during active steering conditions, indicating increased levels of motor-behavioral cognitive control. Finally, we present regression results of both the steering signal and the car position, indicating that a regression of continuous deviations from the road utilizing the EEG might be feasible.
Collapse
Affiliation(s)
- Hannah S. Pulferer
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
| | - Cuntai Guan
- College of Computing and Data Science, Nanyang Technological University, Singapore, Singapore
| | - Gernot R. Müller-Putz
- Institute of Neural Engineering, Graz University of Technology, Graz, Austria
- BioTechMed-Graz, Graz, Austria
| |
Collapse
|
2
|
Li T, Liu P, Gao Y, Ji X, Lin Y. Advancements in Fatigue Detection: Integrating fNIRS and Non-Voluntary Attention Brain Function Experiments. SENSORS (BASEL, SWITZERLAND) 2024; 24:3175. [PMID: 38794028 PMCID: PMC11125156 DOI: 10.3390/s24103175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2024] [Revised: 05/07/2024] [Accepted: 05/14/2024] [Indexed: 05/26/2024]
Abstract
BACKGROUND Driving fatigue is a significant concern in contemporary society, contributing to a considerable number of traffic accidents annually. This study explores novel methods for fatigue detection, aiming to enhance driving safety. METHODS This study utilizes electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) to monitor driver fatigue during simulated driving experiments lasting up to 7 h. RESULTS Analysis reveals a significant correlation between behavioral data and hemodynamic changes in the prefrontal lobe, particularly around the 4 h mark, indicating a critical period for driver performance decline. Despite a small participant cohort, the study's outcomes align closely with established fatigue standards for drivers. CONCLUSIONS By integrating fNIRS into non-voluntary attention brain function experiments, this research demonstrates promising efficacy in accurately detecting driving fatigue. These findings offer insights into fatigue dynamics and have implications for shaping effective safety measures and policies in various industrial settings.
Collapse
Affiliation(s)
- Ting Li
- Institute of Biomedical Engineering, Chinese Academy Medical Sciences & Peking Union Medical College, Tianjin 300192, China; (P.L.); (X.J.)
| | - Peishuai Liu
- Institute of Biomedical Engineering, Chinese Academy Medical Sciences & Peking Union Medical College, Tianjin 300192, China; (P.L.); (X.J.)
| | - Yuan Gao
- Institute of Integrated Circuit Science and Engineering, University of Electronical Science and Technology of China, Chengdu 611731, China;
| | - Xiang Ji
- Institute of Biomedical Engineering, Chinese Academy Medical Sciences & Peking Union Medical College, Tianjin 300192, China; (P.L.); (X.J.)
| | - Yu Lin
- North Carolina State University, Raleigh, NC 27695, USA;
| |
Collapse
|
3
|
Nadalizadeh F, Rajabioun M, Feyzi A. Driving fatigue detection based on brain source activity and ARMA model. Med Biol Eng Comput 2024; 62:1017-1030. [PMID: 38117429 DOI: 10.1007/s11517-023-02983-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 11/28/2023] [Indexed: 12/21/2023]
Abstract
Fatigue among drivers is a significant issue in society, and according to organizational reports, it substantially contributes to accidents. So accurate fatigue detection in drivers plays a crucial role in reducing the number of people fatalities or injured resulting from accidents. Several methods are proposed for fatigue driver recognition among which electroencephalography (EEG) is one. This paper proposed a method for fatigue recognition by EEG signals with extracted features from source and sensor spaces. The proposed method starts with preprocessing by applying filtering and artifact rejection. Then source localization methods are applied to EEG signals for active source extraction. A multivariate autoregressive (MVAR) model is fitted to selected sources, and a dual Kalman filter is applied to estimate the source activity and their relationships. Then multivariate autoregressive moving average (ARMA) is fitted between EEG and source activity signals. Features are extracted from model parameters, source relationship matrix, and wavelet transform of EEG and source activity signals. The novelty of this approach is the use of ARMA model between source activities (as input) and EEG signals (as output) and feature extraction from source relations. Relevant features are selected using a combination of RelifF and neighborhood component analysis (NCA) methods. Three classifiers, namely k-nearest neighbor (KNN), support vector machine (SVM), and naive Bayesian (NB) classifiers, are employed to classify drivers. To improve performance, the final label for fatigue detection is calculated by combining these classifiers using the voting method. The results demonstrate that the proposed method accurately recognizes and classifies fatigued drivers with the ensemble classifiers in comparison with other methods.
Collapse
Affiliation(s)
- Fahimeh Nadalizadeh
- Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran
| | - Mehdi Rajabioun
- Department of Engineering, Mamaghan Branch, Islamic Azad University, Mamaghan, Iran.
| | - Amirreza Feyzi
- Department of Electrical and Computer Engineering, Tabriz University, Tabriz, Iran
| |
Collapse
|
4
|
Wang Q, Smythe D, Cao J, Hu Z, Proctor KJ, Owens AP, Zhao Y. Characterisation of Cognitive Load Using Machine Learning Classifiers of Electroencephalogram Data. SENSORS (BASEL, SWITZERLAND) 2023; 23:8528. [PMID: 37896621 PMCID: PMC10611194 DOI: 10.3390/s23208528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 10/09/2023] [Accepted: 10/16/2023] [Indexed: 10/29/2023]
Abstract
A high cognitive load can overload a person, potentially resulting in catastrophic accidents. It is therefore important to ensure the level of cognitive load associated with safety-critical tasks (such as driving a vehicle) remains manageable for drivers, enabling them to respond appropriately to changes in the driving environment. Although electroencephalography (EEG) has attracted significant interest in cognitive load research, few studies have used EEG to investigate cognitive load in the context of driving. This paper presents a feasibility study on the simulation of various levels of cognitive load through designing and implementing four driving tasks. We employ machine learning-based classification techniques using EEG recordings to differentiate driving conditions. An EEG dataset containing these four driving tasks from a group of 20 participants was collected to investigate whether EEG can be used as an indicator of changes in cognitive load. The collected dataset was used to train four Deep Neural Networks and four Support Vector Machine classification models. The results showed that the best model achieved a classification accuracy of 90.37%, utilising statistical features from multiple frequency bands in 24 EEG channels. Furthermore, the Gamma and Beta bands achieved higher classification accuracy than the Alpha and Theta bands during the analysis. The outcomes of this study have the potential to enhance the Human-Machine Interface of vehicles, contributing to improved safety.
Collapse
Affiliation(s)
- Qi Wang
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Daniel Smythe
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Jun Cao
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Zhilin Hu
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| | - Karl J. Proctor
- Jaguar Land Rover Research, Coventry CV4 7AL, UK; (K.J.P.); (A.P.O.)
| | - Andrew P. Owens
- Jaguar Land Rover Research, Coventry CV4 7AL, UK; (K.J.P.); (A.P.O.)
| | - Yifan Zhao
- School of Aerospace, Transport and Manufacturing, Cranfield University, Cranfield MK43 0AL, UK; (Q.W.); (D.S.); (J.C.); (Z.H.)
| |
Collapse
|
5
|
Peivandi M, Ardabili SZ, Sheykhivand S, Danishvar S. Deep Learning for Detecting Multi-Level Driver Fatigue Using Physiological Signals: A Comprehensive Approach. SENSORS (BASEL, SWITZERLAND) 2023; 23:8171. [PMID: 37837001 PMCID: PMC10574985 DOI: 10.3390/s23198171] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 09/06/2023] [Accepted: 09/11/2023] [Indexed: 10/15/2023]
Abstract
A large share of traffic accidents is related to driver fatigue. In recent years, many studies have been organized in order to diagnose and warn drivers. In this research, a new approach was presented in order to detect multi-level driver fatigue. A multi-level driver tiredness diagnostic database based on physiological signals including ECG, EEG, EMG, and respiratory effort was developed for this aim. The EEG signal was used for processing and other recorded signals were used to confirm the driver's fatigue so that fatigue was not confirmed based on self-report questionnaires. A customized architecture based on adversarial generative networks and convolutional neural networks (end-to-end) was utilized to select/extract features and classify different levels of fatigue. In the customized architecture, with the objective of eliminating uncertainty, type 2 fuzzy sets were used instead of activation functions such as Relu and Leaky Relu, and the performance of each was investigated. The final accuracy obtained in the three scenarios considered, two-level, three-level, and five-level, were 96.8%, 95.1%, and 89.1%, respectively. Given the suggested model's optimal performance, which can identify five various levels of driver fatigue with high accuracy, it can be employed in practical applications of driver fatigue to warn drivers.
Collapse
Affiliation(s)
- Mohammad Peivandi
- Department of Biomedical Engineering, Wayne State University, Detroit, MI 48202, USA;
| | - Sevda Zafarmandi Ardabili
- Electrical and Computer Engineering Department, Southern Methodist University, Dallas, TX 75205, USA;
| | - Sobhan Sheykhivand
- Department of Biomedical Engineering, University of Bonab, Bonab 55517-61167, Iran;
| | - Sebelan Danishvar
- College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK
| |
Collapse
|
6
|
Quiles Pérez M, Martínez Beltrán ET, López Bernal S, Martínez Pérez G, Huertas Celdrán A. Analyzing the impact of Driving tasks when detecting emotions through brain–computer interfaces. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08343-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
AbstractTraffic accidents are the leading cause of death among young people, a problem that today costs an enormous number of victims. Several technologies have been proposed to prevent accidents, being brain–computer interfaces (BCIs) one of the most promising. In this context, BCIs have been used to detect emotional states, concentration issues, or stressful situations, which could play a fundamental role in the road since they are directly related to the drivers’ decisions. However, there is no extensive literature applying BCIs to detect subjects’ emotions in driving scenarios. In such a context, there are some challenges to be solved, such as (i) the impact of performing a driving task on the emotion detection and (ii) which emotions are more detectable in driving scenarios. To improve these challenges, this work proposes a framework focused on detecting emotions using electroencephalography with machine learning and deep learning algorithms. In addition, a use case has been designed where two scenarios are presented. The first scenario consists in listening to sounds as the primary task to perform, while in the second scenario listening to sound becomes a secondary task, being the primary task using a driving simulator. In this way, it is intended to demonstrate whether BCIs are useful in this driving scenario. The results improve those existing in the literature, achieving 99% accuracy for the detection of two emotions (non-stimuli and angry), 93% for three emotions (non-stimuli, angry and neutral) and 75% for four emotions (non-stimuli, angry, neutral and joy).
Collapse
|