1
|
Yadav H, Maini S. Electroencephalogram based brain-computer interface: Applications, challenges, and opportunities. MULTIMEDIA TOOLS AND APPLICATIONS 2023:1-45. [PMID: 37362726 PMCID: PMC10157593 DOI: 10.1007/s11042-023-15653-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 07/17/2022] [Accepted: 04/22/2023] [Indexed: 06/28/2023]
Abstract
Brain-Computer Interfaces (BCI) is an exciting and emerging research area for researchers and scientists. It is a suitable combination of software and hardware to operate any device mentally. This review emphasizes the significant stages in the BCI domain, current problems, and state-of-the-art findings. This article also covers how current results can contribute to new knowledge about BCI, an overview of BCI from its early developments to recent advancements, BCI applications, challenges, and future directions. The authors pointed to unresolved issues and expressed how BCI is valuable for analyzing the human brain. Humans' dependence on machines has led humankind into a new future where BCI can play an essential role in improving this modern world.
Collapse
Affiliation(s)
- Hitesh Yadav
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| | - Surita Maini
- Department of Electrical and Instrumentation Engineering, Sant Longowal Institute of Engineering & Technology, Longowal, Punjab India
| |
Collapse
|
2
|
van Stigt M, Groenendijk E, Marquering H, Coutinho J, Potters W. High performance clean versus artifact dry electrode EEG data classification using Convolutional Neural Network transfer learning. Clin Neurophysiol Pract 2023; 8:88-91. [PMID: 37215683 PMCID: PMC10196906 DOI: 10.1016/j.cnp.2023.04.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 04/06/2023] [Accepted: 04/08/2023] [Indexed: 05/24/2023] Open
Abstract
Objective Convolutional Neural Networks (CNNs) are promising for artifact detection in electroencephalography (EEG) data, but require large amounts of data. Despite increasing use of dry electrodes for EEG data acquisition, dry electrode EEG datasets are sparse. We aim to develop an algorithm for clean versus artifact dry electrode EEG data classification using transfer learning. Methods Dry electrode EEG data were acquired in 13 subjects while physiological and technical artifacts were induced. Data were per 2-second segment labeled as clean or artifact and split in an 80% train and 20% test set. With the train set, we fine-tuned a pre-trained CNN for clean versus artifact wet electrode EEG data classification using 3-fold cross validation. The three fine-tuned CNNs were combined in one final clean versus artifact classification algorithm, in which the majority vote was used for classification. We calculated accuracy, F1-score, precision, and recall of the pre-trained CNN and fine-tuned algorithm when applied to unseen test data. Results The algorithm was trained on 0.40 million and tested on 0.17 million overlapping EEG segments. The pre-trained CNN had a test accuracy of 65.6%. The fine-tuned clean versus artifact classification algorithm had an improved test accuracy of 90.7%, F1-score of 90.2%, precision of 89.1% and recall of 91.2%. Conclusions Despite a relatively small dry electrode EEG dataset, transfer learning enabled development of a high performing CNN-based algorithm for clean versus artifact classification. Significance Development of CNNs for classification of dry electrode EEG data is challenging as dry electrode EEG datasets are sparse. Here, we show that transfer learning can be used to overcome this problem.
Collapse
Affiliation(s)
- M.N. van Stigt
- Amsterdam UMC location University of Amsterdam, Department of Clinical Neurophysiology, Meibergdreef 9, Amsterdam, the Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Neurology, Meibergdreef 9, Amsterdam, the Netherlands
| | - E.A. Groenendijk
- Amsterdam UMC location University of Amsterdam, Department of Clinical Neurophysiology, Meibergdreef 9, Amsterdam, the Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Neurology, Meibergdreef 9, Amsterdam, the Netherlands
| | - H.A. Marquering
- Amsterdam UMC location University of Amsterdam, Department of Biomedical Engineering and Physics, Meibergdreef 9, Amsterdam, the Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Radiology and Nuclear Medicine, Meibergdreef 9, Amsterdam, the Netherlands
| | - J.M. Coutinho
- Amsterdam UMC location University of Amsterdam, Department of Neurology, Meibergdreef 9, Amsterdam, the Netherlands
| | - W.V. Potters
- Amsterdam UMC location University of Amsterdam, Department of Clinical Neurophysiology, Meibergdreef 9, Amsterdam, the Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Neurology, Meibergdreef 9, Amsterdam, the Netherlands
| |
Collapse
|
3
|
Fu R, Li Z, Wang S, Xu D, Huang X, Liang H. EEG-based driver states discrimination by noise fraction analysis and novel clustering algorithm. BIOMED ENG-BIOMED TE 2023:bmt-2022-0395. [PMID: 36848391 DOI: 10.1515/bmt-2022-0395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 02/10/2023] [Indexed: 03/01/2023]
Abstract
Driver states are reported as one of the principal factors in driving safety. Distinguishing the driving driver state based on the artifact-free electroencephalogram (EEG) signal is an effective means, but redundant information and noise will inevitably reduce the signal-to-noise ratio of the EEG signal. This study proposes a method to automatically remove electrooculography (EOG) artifacts by noise fraction analysis. Specifically, multi-channel EEG recordings are collected after the driver experiences a long time driving and after a certain period of rest respectively. Noise fraction analysis is then applied to remove EOG artifacts by separating the multichannel EEG into components by optimizing the signal-to-noise quotient. The representation of data characteristics of the EEG after denoising is found in the Fisher ratio space. Additionally, a novel clustering algorithm is designed to identify denoising EEG by combining cluster ensemble and probability mixture model (CEPM). The EEG mapping plot is used to illustrate the effectiveness and efficiency of noise fraction analysis on the denoising of EEG signals. Adjusted rand index (ARI) and accuracy (ACC) are used to demonstrate clustering performance and precision. The results showed that the noise artifacts in the EEG were removed and the clustering accuracy of all participants was above 90%, resulting in a high driver fatigue recognition rate.
Collapse
Affiliation(s)
- Rongrong Fu
- Department of Electrical Engineering, Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, China
| | - Zheyu Li
- Department of Electrical Engineering, Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, China
| | - Shiwei Wang
- Jiangxi New Energy Technology Institute, Xinyu, China
| | - Dong Xu
- Department of Electrical Engineering, Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, China
| | - Xiaodong Huang
- Department of Electrical Engineering, Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, China
| | - Haifeng Liang
- Department of Electrical Engineering, Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, China
| |
Collapse
|
4
|
Islam MK, Rastegarnia A. Editorial: Recent advances in EEG (non-invasive) based BCI applications. Front Comput Neurosci 2023; 17:1151852. [PMID: 36936191 PMCID: PMC10018154 DOI: 10.3389/fncom.2023.1151852] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Accepted: 02/06/2023] [Indexed: 03/06/2023] Open
Affiliation(s)
- Md. Kafiul Islam
- 1Department of Electrical and Electronic Engineering, Independent University, Dhaka, Bangladesh
- *Correspondence: Md. Kafiul Islam
| | - Amir Rastegarnia
- 2Department of Electrical Engineering, Malayer University, Malayer, Iran
| |
Collapse
|
5
|
Zhao H, Ma J, Zhang Y, Chang R. Mental workload accumulation effect of mobile phone distraction in L2 autopilot mode. Sci Rep 2022; 12:16856. [PMID: 36207431 PMCID: PMC9546873 DOI: 10.1038/s41598-022-17419-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Accepted: 07/25/2022] [Indexed: 11/09/2022] Open
Abstract
As automated vehicles become more common, there is a need for precise measurement and definition of when and in what ways a driver can use a mobile phone in L2 autonomous driving mode, for how long it can be used, the complexity of the call content, and the accumulated mental workload. This study uses a 2 (driving mode) × 2 (call content complexity) × 6 (driving stage) three-factor mixed experimental design to investigate the effect of these factors on the driver's mental workload by measuring the driver's performance on Detection response tasks, pupil diameter, and EEG components in various brain regions in the alpha band. The results showed that drivers' mental workload levels converge between manual and automatic driving modes as the duration of driving increases, regardless of the level of complexity of the mobile phone conversation. This suggests that mobile phone conversations can also disrupt the driver's cognitive resource balance in L2 automatic driving mode, as it increases mental workload while also impairing the normal functioning of brain functions such as cognitive control, problem solving, and judgment, thereby compromising driving safety.
Collapse
Affiliation(s)
- Hongfei Zhao
- School of Psychology, Liaoning Normal University, Dalian, 116029, China
| | - Jinfei Ma
- School of Psychology, Liaoning Normal University, Dalian, 116029, China.
| | - Yijing Zhang
- School of Psychology, Liaoning Normal University, Dalian, 116029, China
| | - Ruosong Chang
- School of Psychology, Liaoning Normal University, Dalian, 116029, China
| |
Collapse
|
6
|
Mughal NE, Khan MJ, Khalil K, Javed K, Sajid H, Naseer N, Ghafoor U, Hong KS. EEG-fNIRS-based hybrid image construction and classification using CNN-LSTM. Front Neurorobot 2022; 16:873239. [PMID: 36119719 PMCID: PMC9472125 DOI: 10.3389/fnbot.2022.873239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 07/20/2022] [Indexed: 11/13/2022] Open
Abstract
The constantly evolving human–machine interaction and advancement in sociotechnical systems have made it essential to analyze vital human factors such as mental workload, vigilance, fatigue, and stress by monitoring brain states for optimum performance and human safety. Similarly, brain signals have become paramount for rehabilitation and assistive purposes in fields such as brain–computer interface (BCI) and closed-loop neuromodulation for neurological disorders and motor disabilities. The complexity, non-stationary nature, and low signal-to-noise ratio of brain signals pose significant challenges for researchers to design robust and reliable BCI systems to accurately detect meaningful changes in brain states outside the laboratory environment. Different neuroimaging modalities are used in hybrid settings to enhance accuracy, increase control commands, and decrease the time required for brain activity detection. Functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) measure the hemodynamic and electrical activity of the brain with a good spatial and temporal resolution, respectively. However, in hybrid settings, where both modalities enhance the output performance of BCI, their data compatibility due to the huge discrepancy between their sampling rate and the number of channels remains a challenge for real-time BCI applications. Traditional methods, such as downsampling and channel selection, result in important information loss while making both modalities compatible. In this study, we present a novel recurrence plot (RP)-based time-distributed convolutional neural network and long short-term memory (CNN-LSTM) algorithm for the integrated classification of fNIRS EEG for hybrid BCI applications. The acquired brain signals are first projected into a non-linear dimension with RPs and fed into the CNN to extract essential features without performing any downsampling. Then, LSTM is used to learn the chronological features and time-dependence relation to detect brain activity. The average accuracies achieved with the proposed model were 78.44% for fNIRS, 86.24% for EEG, and 88.41% for hybrid EEG-fNIRS BCI. Moreover, the maximum accuracies achieved were 85.9, 88.1, and 92.4%, respectively. The results confirm the viability of the RP-based deep-learning algorithm for successful BCI systems.
Collapse
Affiliation(s)
- Nabeeha Ehsan Mughal
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Muhammad Jawad Khan
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
- National Center of Artificial Intelligence (NCAI) – NUST, Islamabad, Pakistan
| | - Khurram Khalil
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Kashif Javed
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
| | - Hasan Sajid
- School of Mechanical and Manufacturing Engineering, National University of Sciences and Technology (NUST), Islamabad, Pakistan
- National Center of Artificial Intelligence (NCAI) – NUST, Islamabad, Pakistan
| | - Noman Naseer
- Department of Mechatronics and Biomedical Engineering, Air University, Islamabad, Pakistan
| | - Usman Ghafoor
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
| | - Keum-Shik Hong
- School of Mechanical Engineering, Pusan National University, Busan, South Korea
- *Correspondence: Keum-Shik Hong
| |
Collapse
|
7
|
Dora M, Holcman D. Adaptive single-channel EEG artifact removal for real-time clinical monitoring. IEEE Trans Neural Syst Rehabil Eng 2022; 30:286-295. [PMID: 35085086 DOI: 10.1109/tnsre.2022.3147072] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
OBJECTIVE Electroencephalography (EEG) has become very common in clinical practice due to its relatively low cost, ease of installation, non-invasiveness, and good temporal resolution. Portable EEG devices are increasingly popular in clinical monitoring applications such as sleep scoring or anesthesia monitoring. In these situations, for reasons of speed and simplicity only few electrodes are used and contamination of the EEG signal by artifacts is inevitable. Visual inspection and manual removal of artifacts is often not possible, especially in real-time applications. Our goal is to develop a flexible technique to remove EEG artifacts in these contexts with minimal supervision. METHODS We propose here a new wavelet-based method which allows to remove artifacts from single-channel EEGs. The method is based on a data-driven renormalization of the wavelet components and is capable of adaptively attenuate artifacts of different nature. We benchmark our method against alternative artifact removal techniques. RESULTS We assessed the performance of the proposed method on publicly available datasets comprising ocular, muscular, and movement artifacts. The proposed method shows superior performances on different kinds of artifacts and signal-to-noise levels. Finally, we present an application of our method to the monitoring of general anesthesia. CONCLUSIONS We show that our method can successfully attenuate various types of artifacts in single-channel EEG. SIGNIFICANCE Thanks to its data-driven approach and low computational cost, the proposed method provides a valuable tool to remove artifacts in real-time EEG applications with few electrodes, such as monitoring in special care units.
Collapse
|
8
|
Electrophysiological frequency domain analysis of driver passive fatigue under automated driving conditions. Sci Rep 2021; 11:20348. [PMID: 34645882 PMCID: PMC8514533 DOI: 10.1038/s41598-021-99680-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2021] [Accepted: 09/27/2021] [Indexed: 11/08/2022] Open
Abstract
With the continuous improvement of automated vehicles, researchers have found that automated driving is more likely to cause passive fatigue. To explore the impact of automation and scenario complexity on the passive fatigue of a driver, we collected electroencephalography (EEG), detection-response task (DRT) performance, and the subjective report scores of 48 drivers. We found that in automated driving under monotonic conditions, after 40 min, the alpha power of the driver’s EEG indicators increased significantly, the accuracy of the detection reaction task decreased, and the reaction time became slower. The receiver characteristic curve was used to calculate the critical threshold of the alpha power during passive fatigue. The determination of the threshold further clarifies the occurrence time and physiological characteristics of passive fatigue and improves the passive fatigue theory.
Collapse
|