1
|
Li R, Bai D, Li Z, Yang S, Liu W, Zhang Y, Zhou J, Luo J, Wang W. The SSHVEP Paradigm-Based Brain Controlled Method for Grasping Robot Using MVMD Combined CNN Model. IEEE Trans Neural Syst Rehabil Eng 2024; 32:2564-2578. [PMID: 38980788 DOI: 10.1109/tnsre.2024.3425636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/11/2024]
Abstract
In recent years, the steady-state visual evoked potentials (SSVEP) based brain control method has been employed to help people with disabilities because of its advantages of high information transmission rate and low training time. However, the existing SSVEP brain control methods cannot adapt to dynamic or unstructured environments. Moreover, the recognition accuracy from the conventional decoding algorithm still needs to improve. To address the above problems, this study proposed a steady-state hybrid visual evoked potentials (SSHVEP) paradigm using the grasping targets in their environment to improve the connection between the subjects' and their dynamic environments. Moreover, a novel EEG decoding method, using the multivariate variational mode decomposition (MVMD) algorithm for adaptive sub-band division and convolutional neural network (CNN) for target recognition, was applied to improve the decoding accuracy of the SSHVEPs. 18 subjects participated in the offline and online experiments. The offline accuracy across 18 subjects by the 9-target SSHVEP paradigm was up to 95.41 ± 2.70 %, which is a 5.80% improvement compared to the conventional algorithm. To further validate the performance of the proposed method, the brain-controlled grasping robot system using the SSHVEP paradigm was built. The average accuracy reached 93.21 ± 10.18 % for the online experiment. All the experimental results demonstrated the effectiveness of the brain-computer interaction method based on the SSHVEP paradigm and the MVMD combined CNN algorithm studied in this paper.
Collapse
|
2
|
Wang H, Wang Z, Sun Y, Yuan Z, Xu T, Li J. A Cascade xDAWN EEGNet Structure for Unified Visual-Evoked Related Potential Detection. IEEE Trans Neural Syst Rehabil Eng 2024; 32:2270-2280. [PMID: 38885099 DOI: 10.1109/tnsre.2024.3415474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/20/2024]
Abstract
Visual-based brain-computer interface (BCI) enables people to communicate with others by spelling words from the brain and helps professionals recognize targets in large numbers of images. P300 signals evoked by different types of stimuli, such as words or images, may vary significantly in terms of both amplitude and latency. A unified approach is required to detect variable P300 signals, which facilitates BCI applications, as well as deepens the understanding of the P300 generation mechanism. In this study, our proposed approach involves a cascade network structure that combines xDAWN and classical EEGNet techniques. This network is designed to classify target and non-target stimuli in both P300 speller and rapid serial visual presentation (RSVP) paradigms. The proposed approach is capable of recognizing more symbols with fewer repetitions (up to 5 rounds) compared to other models while possessing a better information transfer rate (ITR) as demonstrated on Dataset II (17.22 bits/min in the second repetition round) of BCI Competition III. Additionally, our approach has the highest unweighted average recall (UAR) performance for both 5 Hz ( 0.8134±0.0259 ) and 20 Hz ( 0.6527±0.0321 ) RSVP. The results show that the cascade network structure has better performance between both the P300 Speller and RSVP paradigms, manifesting that such a cascade structure is robust enough for dealing with P300-related signals (source code is available at https://github.com/embneural/Cascade-xDAWN-EEGNet-for-ERP-Detection).
Collapse
|
3
|
Eidel M, Pfeiffer M, Ziebell P, Kübler A. Recording the tactile P300 with the cEEGrid for potential use in a brain-computer interface. Front Hum Neurosci 2024; 18:1371631. [PMID: 38957693 PMCID: PMC11218745 DOI: 10.3389/fnhum.2024.1371631] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 05/27/2024] [Indexed: 07/04/2024] Open
Abstract
Brain-computer interfaces (BCIs) are scientifically well established, but they rarely arrive in the daily lives of potential end-users. This could be in part because electroencephalography (EEG), a prevalent method to acquire brain activity for BCI operation, is considered too impractical to be applied in daily life of end-users with physical impairment as an assistive device. Hence, miniaturized EEG systems such as the cEEGrid have been developed. While they promise to be a step toward bridging the gap between BCI development, lab demonstrations, and home use, they still require further validation. Encouragingly, the cEEGrid has already demonstrated its ability to record visually and auditorily evoked event-related potentials (ERP), which are important as input signal for many BCIs. With this study, we aimed at evaluating the cEEGrid in the context of a BCI based on tactually evoked ERPs. To compare the cEEGrid with a conventional scalp EEG, we recorded brain activity with both systems simultaneously. Forty healthy participants were recruited to perform a P300 oddball task based on vibrotactile stimulation at four different positions. This tactile paradigm has been shown to be feasible for BCI repeatedly but has never been tested with the cEEGrid. We found distinct P300 deflections in the cEEGrid data, particularly at vertical bipolar channels. With an average of 63%, the cEEGrid classification accuracy was significantly above the chance level (25%) but significantly lower than the 81% reached with the EEG cap. Likewise, the P300 amplitude was significantly lower (cEEGrid R2-R7: 1.87 μV, Cap Cz: 3.53 μV). These results indicate that a tactile BCI using the cEEGrid could potentially be operated, albeit with lower efficiency. Additionally, participants' somatosensory sensitivity was assessed, but no correlation to the accuracy of either EEG system was shown. Our research contributes to the growing amount of literature comparing the cEEGrid to conventional EEG systems and provides first evidence that the tactile P300 can be recorded behind the ear. A BCI based on a thus simplified EEG system might be more readily accepted by potential end-users, provided the accuracy can be substantially increased, e.g., by training and improved classification.
Collapse
Affiliation(s)
- M. Eidel
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | | | | | | |
Collapse
|
4
|
Ron-Angevin R, Fernández-Rodríguez Á, Velasco-Álvarez F, Lespinet-Najib V, André JM. Evaluation of Different Types of Stimuli in an Event-Related Potential-Based Brain-Computer Interface Speller under Rapid Serial Visual Presentation. SENSORS (BASEL, SWITZERLAND) 2024; 24:3315. [PMID: 38894107 PMCID: PMC11174573 DOI: 10.3390/s24113315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Revised: 05/10/2024] [Accepted: 05/21/2024] [Indexed: 06/21/2024]
Abstract
Rapid serial visual presentation (RSVP) is currently a suitable gaze-independent paradigm for controlling visual brain-computer interfaces (BCIs) based on event-related potentials (ERPs), especially for users with limited eye movement control. However, unlike gaze-dependent paradigms, gaze-independent ones have received less attention concerning the specific choice of visual stimuli that are used. In gaze-dependent BCIs, images of faces-particularly those tinted red-have been shown to be effective stimuli. This study aims to evaluate whether the colour of faces used as visual stimuli influences ERP-BCI performance under RSVP. Fifteen participants tested four conditions that varied only in the visual stimulus used: grey letters (GL), red famous faces with letters (RFF), green famous faces with letters (GFF), and blue famous faces with letters (BFF). The results indicated significant accuracy differences only between the GL and GFF conditions, unlike prior gaze-dependent studies. Additionally, GL achieved higher comfort ratings compared with other face-related conditions. This study highlights that the choice of stimulus type impacts both performance and user comfort, suggesting implications for future ERP-BCI designs for users requiring gaze-independent systems.
Collapse
Affiliation(s)
- Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain; (Á.F.-R.); (F.V.-Á.)
| | - Álvaro Fernández-Rodríguez
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain; (Á.F.-R.); (F.V.-Á.)
| | - Francisco Velasco-Álvarez
- Departamento de Tecnología Electrónica, Instituto Universitario de Investigación en Telecomunicación de la Universidad de Málaga (TELMA), Universidad de Málaga, 29071 Malaga, Spain; (Á.F.-R.); (F.V.-Á.)
| | - Véronique Lespinet-Najib
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Bordeaux, France; (V.L.-N.); (J.-M.A.)
| | - Jean-Marc André
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Bordeaux, France; (V.L.-N.); (J.-M.A.)
| |
Collapse
|
5
|
Proverbio AM, Cesati F. Neural correlates of recalled sadness, joy, and fear states: a source reconstruction EEG study. Front Psychiatry 2024; 15:1357770. [PMID: 38638416 PMCID: PMC11024723 DOI: 10.3389/fpsyt.2024.1357770] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Accepted: 03/18/2024] [Indexed: 04/20/2024] Open
Abstract
Introduction The capacity to understand the others' emotional states, particularly if negative (e.g. sadness or fear), underpins the empathic and social brain. Patients who cannot express their emotional states experience social isolation and loneliness, exacerbating distress. We investigated the feasibility of detecting non-invasive scalp-recorded electrophysiological signals that correspond to recalled emotional states of sadness, fear, and joy for potential classification. Methods The neural activation patterns of 20 healthy and right-handed participants were studied using an electrophysiological technique. Analyses were focused on the N400 component of Event-related potentials (ERPs) recorded during silent recall of subjective emotional states; Standardized weighted Low-resolution Electro-magnetic Tomography (swLORETA) was employed for source reconstruction. The study classified individual patterns of brain activation linked to the recollection of three distinct emotional states into seven regions of interest (ROIs). Results Statistical analysis (ANOVA) of the individual magnitude values revealed the existence of a common emotional circuit, as well as distinct brain areas that were specifically active during recalled sad, happy and fearful states. In particular, the right temporal and left superior frontal areas were more active for sadness, the left limbic region for fear, and the right orbitofrontal cortex for happy affective states. Discussion In conclusion, this study successfully demonstrated the feasibility of detecting scalp-recorded electrophysiological signals corresponding to internal and subjective affective states. These findings contribute to our understanding of the emotional brain, and have potential applications for future BCI classification and identification of emotional states in LIS patients who may be unable to express their emotions, thus helping to alleviate social isolation and sense of loneliness.
Collapse
Affiliation(s)
- Alice Mado Proverbio
- Cognitive Electrophysiology Lab, Department of Psychology, University of Milano-Bicocca, Milan, Italy
- NEURO-MI Milan Center for Neuroscience, Milan, Italy
| | - Federico Cesati
- Cognitive Electrophysiology Lab, Department of Psychology, University of Milano-Bicocca, Milan, Italy
| |
Collapse
|
6
|
Li M, Li J, Song Z, Deng H, Xu J, Xu G, Liao W. EEGNet-based multi-source domain filter for BCI transfer learning. Med Biol Eng Comput 2024; 62:675-686. [PMID: 37982955 DOI: 10.1007/s11517-023-02967-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 11/06/2023] [Indexed: 11/21/2023]
Abstract
Deep learning has great potential on decoding EEG in brain-computer interface. While common deep learning algorithms cannot directly train models with data from multiple individuals because of the inter-individual differences in EEG. Collecting enough data for each subject to satisfy the training of deep learning would result in an increase in training cost. This study proposes a novel transfer learning, EEGNet-based multi-source domain filter for transfer learning (EEGNet-MDFTL), to reduce the amount of training data and improve the performance of BCI. The EEGNet-MDFTL uses bagging ensemble learning to learn domain-invariant features from the multi-source domain and utilizes model loss value to filter the multi-source domain. Compared with baseline methods, the accuracy of the EEGNet-MDFTL reaches 91.96%, higher than two state-of-the-art methods, which demonstrates source domain filter can select similar source domains to improve the accuracy of the model, and remains a high level even when the data amount is reduced to 1/8, proving that ensemble learning learns enough domain invariant features from the multi-source domain to make the model insensitive to data amount. The proposed EEGNet-MDFTL is effective in improving the decoding performance with a small amount of data, which is helpful to save the BCI training cost.
Collapse
Affiliation(s)
- Mengfan Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Science and Biomedical Engineering, Hebei University of Technology, Tianjin, China.
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin, China.
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Tianjin, China.
| | - Jundi Li
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Science and Biomedical Engineering, Hebei University of Technology, Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Tianjin, China
| | - Zhiyong Song
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Science and Biomedical Engineering, Hebei University of Technology, Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Tianjin, China
| | - Haodong Deng
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Science and Biomedical Engineering, Hebei University of Technology, Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Tianjin, China
| | - Jiaming Xu
- Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Guizhi Xu
- State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Science and Biomedical Engineering, Hebei University of Technology, Tianjin, China
- Hebei Key Laboratory of Bioelectromagnetics and Neuroengineering, Tianjin, China
- Tianjin Key Laboratory of Bioelectromagnetic Technology and Intelligent Health, Tianjin, China
| | - Wenzhe Liao
- School of Artificial Intelligence, Hebei University of Technology, Tianjin, China
| |
Collapse
|
7
|
Mimnaugh KJ, Center EG, Suomalainen M, Becerra I, Lozano E, Murrieta-Cid R, Ojala T, LaValle SM, Federmeier KD. Virtual Reality Sickness Reduces Attention During Immersive Experiences. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; 29:4394-4404. [PMID: 37788212 DOI: 10.1109/tvcg.2023.3320222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
In this paper, we show that Virtual Reality (VR) sickness is associated with a reduction in attention, which was detected with the P3b Event-Related Potential (ERP) component from electroencephalography (EEG) measurements collected in a dual-task paradigm. We hypothesized that sickness symptoms such as nausea, eyestrain, and fatigue would reduce the users' capacity to pay attention to tasks completed in a virtual environment, and that this reduction in attention would be dynamically reflected in a decrease of the P3b amplitude while VR sickness was experienced. In a user study, participants were taken on a tour through a museum in VR along paths with varying amounts of rotation, shown previously to cause different levels of VR sickness. While paying attention to the virtual museum (the primary task), participants were asked to silently count tones of a different frequency (the secondary task). Control measurements for comparison against the VR sickness conditions were taken when the users were not wearing the Head-Mounted Display (HMD) and while they were immersed in VR but not moving through the environment. This exploratory study shows, across multiple analyses, that the effect mean amplitude of the P3b collected during the task is associated with both sickness severity measured after the task with a questionnaire (SSQ) and with the number of counting errors on the secondary task. Thus, VR sickness may impair attention and task performance, and these changes in attention can be tracked with ERP measures as they happen, without asking participants to assess their sickness symptoms in the moment.
Collapse
|
8
|
Peksa J, Mamchur D. State-of-the-Art on Brain-Computer Interface Technology. SENSORS (BASEL, SWITZERLAND) 2023; 23:6001. [PMID: 37447849 DOI: 10.3390/s23136001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Revised: 06/23/2023] [Accepted: 06/26/2023] [Indexed: 07/15/2023]
Abstract
This paper provides a comprehensive overview of the state-of-the-art in brain-computer interfaces (BCI). It begins by providing an introduction to BCIs, describing their main operation principles and most widely used platforms. The paper then examines the various components of a BCI system, such as hardware, software, and signal processing algorithms. Finally, it looks at current trends in research related to BCI use for medical, educational, and other purposes, as well as potential future applications of this technology. The paper concludes by highlighting some key challenges that still need to be addressed before widespread adoption can occur. By presenting an up-to-date assessment of the state-of-the-art in BCI technology, this paper will provide valuable insight into where this field is heading in terms of progress and innovation.
Collapse
Affiliation(s)
- Janis Peksa
- Department of Information Technologies, Turiba University, Graudu Street 68, LV-1058 Riga, Latvia
- Institute of Information Technology, Riga Technical University, Kalku Street 1, LV-1658 Riga, Latvia
| | - Dmytro Mamchur
- Department of Information Technologies, Turiba University, Graudu Street 68, LV-1058 Riga, Latvia
- Computer Engineering and Electronics Department, Kremenchuk Mykhailo Ostrohradskyi National University, Pershotravneva 20, 39600 Kremenchuk, Ukraine
| |
Collapse
|
9
|
Long T, Wan M, Jian W, Dai H, Nie W, Xu J. Application of multi-task transfer learning: The combination of EA and optimized subband regularized CSP to classification of 8-channel EEG signals with small dataset. Front Hum Neurosci 2023; 17:1143027. [PMID: 37056962 PMCID: PMC10089123 DOI: 10.3389/fnhum.2023.1143027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 03/03/2023] [Indexed: 03/30/2023] Open
Abstract
IntroductionThe volume conduction effect and high dimensional characteristics triggered by the excessive number of channels of EEG cap-acquired signals in BCI systems can increase the difficulty of classifying EEG signals and the lead time of signal acquisition. We aim to combine transfer learning to decode EEG signals in the few-channel case, improve the classification performance of the motor imagery BCI system across subject cases, reduce the cost of signal acquisition performed by the BCI system, and improve the usefulness of the system.MethodsDataset2a from BCI CompetitionIV(2008) was used as Dataset1, and our team's self-collected dataset was used as Dataset2. Dataset1 acquired EEG signals from 9 subjects using a 22-channel device with a sampling frequency of 250 Hz. Dataset2 acquired EEG signals from 10 healthy subjects (8 males and 2 females; age distribution between 21-30 years old; mean age 25 years old) using an 8-channel system with a sampling frequency of 1000 Hz. We introduced EA in the data preprocessing process to reduce the signal differences between subjects and proposed VFB-RCSP in combination with RCSP and FBCSP to optimize the effect of feature extraction.ResultsExperiments were conducted on Dataset1 with EEG data containing only 8 channels and achieved an accuracy of 78.01 and a kappa coefficient of 0.54. The accuracy exceeded most of the other methods proposed in recent years, even though the number of channels used was significantly reduced. On Dataset 2, an accuracy of 59.77 and a Kappa coefficient of 0.34 were achieved, which is a significant improvement compared to other poorly improved classical protocols.DiscussionOur work effectively improves the classification of few-channel EEG data. It overcomes the dependence of existing algorithms on the number of channels, the number of samples, and the frequency band, which is significant for reducing the complexity of BCI models and improving the user-friendliness of BCI systems.
Collapse
Affiliation(s)
- Taixue Long
- The Second Affiliated Hospital, Nanchang University, Nanchang, Jiangxi, China
- Information Engineering School, Nanchang University, Nanchang, Jiangxi, China
| | - Min Wan
- The Second Affiliated Hospital, Nanchang University, Nanchang, Jiangxi, China
| | - Wenjuan Jian
- Information Engineering School, Nanchang University, Nanchang, Jiangxi, China
- *Correspondence: Wenjuan Jian
| | - Honghui Dai
- Information Engineering School, Nanchang University, Nanchang, Jiangxi, China
| | - Wenbing Nie
- The Army Infantry College of PLA, Nanchang, Jiangxi, China
| | - Jianzhong Xu
- The Army Infantry College of PLA, Nanchang, Jiangxi, China
| |
Collapse
|
10
|
Fernández-Rodríguez Á, Darves-Bornoz A, Velasco-Álvarez F, Ron-Angevin R. Effect of Stimulus Size in a Visual ERP-Based BCI under RSVP. SENSORS (BASEL, SWITZERLAND) 2022; 22:9505. [PMID: 36502205 PMCID: PMC9741214 DOI: 10.3390/s22239505] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 11/23/2022] [Accepted: 12/02/2022] [Indexed: 06/17/2023]
Abstract
Rapid serial visual presentation (RSVP) is currently one of the most suitable paradigms for use with a visual brain-computer interface based on event-related potentials (ERP-BCI) by patients with a lack of ocular motility. However, gaze-independent paradigms have not been studied as closely as gaze-dependent ones, and variables such as the sizes of the stimuli presented have not yet been explored under RSVP. Hence, the aim of the present work is to assess whether stimulus size has an impact on ERP-BCI performance under the RSVP paradigm. Twelve participants tested the ERP-BCI under RSVP using three different stimulus sizes: small (0.1 × 0.1 cm), medium (1.9 × 1.8 cm), and large (20.05 × 19.9 cm) at 60 cm. The results showed significant differences in accuracy between the conditions; the larger the stimulus, the better the accuracy obtained. It was also shown that these differences were not due to incorrect perception of the stimuli since there was no effect from the size in a perceptual discrimination task. The present work therefore shows that stimulus size has an impact on the performance of an ERP-BCI under RSVP. This finding should be considered by future ERP-BCI proposals aimed at users who need gaze-independent systems.
Collapse
Affiliation(s)
| | | | | | - Ricardo Ron-Angevin
- Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Malaga, Spain
| |
Collapse
|
11
|
Brain-computer interface (BCI)-generated speech to control domotic devices. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.08.068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
12
|
Li R, Liu D, Li Z, Liu J, Zhou J, Liu W, Liu B, Fu W, Alhassan AB. A novel EEG decoding method for a facial-expression-based BCI system using the combined convolutional neural network and genetic algorithm. Front Neurosci 2022; 16:988535. [PMID: 36177358 PMCID: PMC9513431 DOI: 10.3389/fnins.2022.988535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 08/17/2022] [Indexed: 11/19/2022] Open
Abstract
Multiple types of brain-control systems have been applied in the field of rehabilitation. As an alternative scheme for balancing user fatigue and the classification accuracy of brain–computer interface (BCI) systems, facial-expression-based brain control technologies have been proposed in the form of novel BCI systems. Unfortunately, existing machine learning algorithms fail to identify the most relevant features of electroencephalogram signals, which further limits the performance of the classifiers. To address this problem, an improved classification method is proposed for facial-expression-based BCI (FE-BCI) systems, using a convolutional neural network (CNN) combined with a genetic algorithm (GA). The CNN was applied to extract features and classify them. The GA was used for hyperparameter selection to extract the most relevant parameters for classification. To validate the superiority of the proposed algorithm used in this study, various experimental performance results were systematically evaluated, and a trained CNN-GA model was constructed to control an intelligent car in real time. The average accuracy across all subjects was 89.21 ± 3.79%, and the highest accuracy was 97.71 ± 2.07%. The superior performance of the proposed algorithm was demonstrated through offline and online experiments. The experimental results demonstrate that our improved FE-BCI system outperforms the traditional methods.
Collapse
Affiliation(s)
- Rui Li
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
- Xi'an People's Hospital, Xi'an, China
- *Correspondence: Rui Li
| | - Di Liu
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Zhijun Li
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Jinli Liu
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Jincao Zhou
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Weiping Liu
- Xi'an People's Hospital, Xi'an, China
- Weiping Liu
| | - Bo Liu
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Weiping Fu
- School of Mechanical and Instrumental Engineering, Xi'an University of Technology, Xi'an, China
| | - Ahmad Bala Alhassan
- Department of Electrical and Information Technology, King Mongkut's University of Technology, Bangkok, Thailand
| |
Collapse
|
13
|
Leoni J, Strada SC, Tanelli M, Brusa A, Proverbio AM. Single-trial stimuli classification from detected P300 for augmented Brain–Computer Interface: A deep learning approach. MACHINE LEARNING WITH APPLICATIONS 2022. [DOI: 10.1016/j.mlwa.2022.100393] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
|
14
|
Spataro R, Xu Y, Xu R, Mandalà G, Allison BZ, Ortner R, Heilinger A, La Bella V, Guger C. How brain-computer interface technology may improve the diagnosis of the disorders of consciousness: A comparative study. Front Neurosci 2022; 16:959339. [PMID: 36033632 PMCID: PMC9404379 DOI: 10.3389/fnins.2022.959339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 07/18/2022] [Indexed: 01/18/2023] Open
Abstract
Objective Clinical assessment of consciousness relies on behavioural assessments, which have several limitations. Hence, disorder of consciousness (DOC) patients are often misdiagnosed. In this work, we aimed to compare the repetitive assessment of consciousness performed with a clinical behavioural and a Brain-Computer Interface (BCI) approach. Materials and methods For 7 weeks, sixteen DOC patients participated in weekly evaluations using both the Coma Recovery Scale-Revised (CRS-R) and a vibrotactile P300 BCI paradigm. To use the BCI, patients had to perform an active mental task that required detecting specific stimuli while ignoring other stimuli. We analysed the reliability and the efficacy in the detection of command following resulting from the two methodologies. Results Over repetitive administrations, the BCI paradigm detected command following before the CRS-R in seven patients. Four clinically unresponsive patients consistently showed command following during the BCI assessments. Conclusion Brain-Computer Interface active paradigms might contribute to the evaluation of the level of consciousness, increasing the diagnostic precision of the clinical bedside approach. Significance The integration of different diagnostic methods leads to a better knowledge and care for the DOC.
Collapse
Affiliation(s)
- Rossella Spataro
- IRCCS Centro Neurolesi Bonino Pulejo, Palermo, Italy
- ALS Clinical Research Center, University of Palermo, Palermo, Italy
- *Correspondence: Rossella Spataro,
| | - Yiyan Xu
- ALS Clinical Research Center, University of Palermo, Palermo, Italy
| | - Ren Xu
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
| | - Giorgio Mandalà
- Rehabilitation Unit, Ospedale Buccheri La Ferla, Palermo, Italy
| | - Brendan Z. Allison
- Cognitive Science Department, University of California, San Diego, San Diego, United States
| | - Rupert Ortner
- g.tec Medical Engineering Spain S.L., Barcelona, Spain
| | | | | | - Christoph Guger
- g.tec Medical Engineering GmbH, Schiedlberg, Austria
- g.tec Medical Engineering Spain S.L., Barcelona, Spain
| |
Collapse
|
15
|
Riccio A, Schettini F, Galiotta V, Giraldi E, Grasso MG, Cincotti F, Mattia D. Usability of a Hybrid System Combining P300-Based Brain-Computer Interface and Commercial Assistive Technologies to Enhance Communication in People With Multiple Sclerosis. Front Hum Neurosci 2022; 16:868419. [PMID: 35721361 PMCID: PMC9204311 DOI: 10.3389/fnhum.2022.868419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2022] [Accepted: 04/26/2022] [Indexed: 11/25/2022] Open
Abstract
Brain-computer interface (BCI) can provide people with motor disabilities with an alternative channel to access assistive technology (AT) software for communication and environmental interaction. Multiple sclerosis (MS) is a chronic disease of the central nervous system that mostly starts in young adulthood and often leads to a long-term disability, possibly exacerbated by the presence of fatigue. Patients with MS have been rarely considered as potential BCI end-users. In this pilot study, we evaluated the usability of a hybrid BCI (h-BCI) system that enables both a P300-based BCI and conventional input devices (i.e., muscular dependent) to access mainstream applications through the widely used AT software for communication “Grid 3.” The evaluation was performed according to the principles of the user-centered design (UCD) with the aim of providing patients with MS with an alternative control channel (i.e., BCI), potentially less sensitive to fatigue. A total of 13 patients with MS were enrolled. In session I, participants were presented with a widely validated P300-based BCI (P3-speller); in session II, they had to operate Grid 3 to access three mainstream applications with (1) an AT conventional input device and (2) the h-BCI. Eight patients completed the protocol. Five out of eight patients with MS were successfully able to access the Grid 3 via the BCI, with a mean online accuracy of 83.3% (± 14.6). Effectiveness (online accuracy), satisfaction, and workload were comparable between the conventional AT inputs and the BCI channel in controlling the Grid 3. As expected, the efficiency (time for correct selection) resulted to be significantly lower for the BCI with respect to the AT conventional channels (Z = 0.2, p < 0.05). Although cautious due to the limited sample size, these preliminary findings indicated that the BCI control channel did not have a detrimental effect with respect to conventional AT channels on the ability to operate an AT software (Grid 3). Therefore, we inferred that the usability of the two access modalities was comparable. The integration of BCI with commercial AT input devices to access a widely used AT software represents an important step toward the introduction of BCIs into the AT centers’ daily practice.
Collapse
Affiliation(s)
- Angela Riccio
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
- *Correspondence: Angela Riccio,
| | - Francesca Schettini
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Valentina Galiotta
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | - Enrico Giraldi
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
| | | | - Febo Cincotti
- Department of Computer, Control and Management Engineering Antonio Ruberti, Sapienza University of Rome, Rome, Italy
| | - Donatella Mattia
- Neuroelectric Imaging and BCI Lab, Fondazione Santa Lucia (IRCCS), Rome, Italy
- Servizio Ausilioteca per la Riabilitazione Assistita con Tecnologia, Fondazione Santa Lucia (IRCCS), Rome, Italy
| |
Collapse
|
16
|
Han Y, Ziebell P, Riccio A, Halder S. Two sides of the same coin: adaptation of BCIs to internal states with user-centered design and electrophysiological features. BRAIN-COMPUTER INTERFACES 2022. [DOI: 10.1080/2326263x.2022.2041294] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Affiliation(s)
- Yiyuan Han
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| | - Philipp Ziebell
- Institute of Psychology, University of Würzburg, Würzburg, Germany
| | - Angela Riccio
- Neuroelectrical Imaging and Brain Computer Interface Laboratory,Fondazione Santa Lucia, Irccs, Rome, Italy
| | - Sebastian Halder
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
| |
Collapse
|
17
|
Bruera A, Poesio M. Exploring the Representations of Individual Entities in the Brain Combining EEG and Distributional Semantics. Front Artif Intell 2022; 5:796793. [PMID: 35280237 PMCID: PMC8905499 DOI: 10.3389/frai.2022.796793] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Accepted: 01/25/2022] [Indexed: 11/23/2022] Open
Abstract
Semantic knowledge about individual entities (i.e., the referents of proper names such as Jacinta Ardern) is fine-grained, episodic, and strongly social in nature, when compared with knowledge about generic entities (the referents of common nouns such as politician). We investigate the semantic representations of individual entities in the brain; and for the first time we approach this question using both neural data, in the form of newly-acquired EEG data, and distributional models of word meaning, employing them to isolate semantic information regarding individual entities in the brain. We ran two sets of analyses. The first set of analyses is only concerned with the evoked responses to individual entities and their categories. We find that it is possible to classify them according to both their coarse and their fine-grained category at appropriate timepoints, but that it is hard to map representational information learned from individuals to their categories. In the second set of analyses, we learn to decode from evoked responses to distributional word vectors. These results indicate that such a mapping can be learnt successfully: this counts not only as a demonstration that representations of individuals can be discriminated in EEG responses, but also as a first brain-based validation of distributional semantic models as representations of individual entities. Finally, in-depth analyses of the decoder performance provide additional evidence that the referents of proper names and categories have little in common when it comes to their representation in the brain.
Collapse
Affiliation(s)
- Andrea Bruera
- Cognitive Science Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London, United Kingdom
| | | |
Collapse
|
18
|
Li M, Zhang P, Yang G, Xu G, Guo M, Liao W. A fisher linear discriminant analysis classifier fused with naïve Bayes for simultaneous detection in an asynchronous brain-computer interface. J Neurosci Methods 2022; 371:109496. [DOI: 10.1016/j.jneumeth.2022.109496] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2021] [Revised: 01/10/2022] [Accepted: 02/06/2022] [Indexed: 11/16/2022]
|
19
|
|
20
|
Zisk AH, Borgheai SB, McLinden J, Deligani RJ, Shahriari Y. Improving longitudinal P300-BCI performance for people with ALS using a data augmentation and jitter correction approach. BRAIN-COMPUTER INTERFACES 2021. [DOI: 10.1080/2326263x.2021.2014678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Alyssa Hillary Zisk
- Interdisciplinary Neuroscience Program, University of Rhode Island, Kingston, RI, –USA
| | - Seyyed Bahram Borgheai
- Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island, Fascitelli Center for Advanced Engineering, –USA Kingston, RI, USA
| | - John McLinden
- Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island, Fascitelli Center for Advanced Engineering, –USA Kingston, RI, USA
| | - Roohollah Jafari Deligani
- Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island, Fascitelli Center for Advanced Engineering, –USA Kingston, RI, USA
| | - Yalda Shahriari
- Interdisciplinary Neuroscience Program, University of Rhode Island, Kingston, RI, –USA
- Department of Electrical, Computer, and Biomedical Engineering, University of Rhode Island, Fascitelli Center for Advanced Engineering, –USA Kingston, RI, USA
| |
Collapse
|
21
|
Petit J, Rouillard J, Cabestaing F. EEG-based brain-computer interfaces exploiting steady-state somatosensory-evoked potentials: a literature review. J Neural Eng 2021; 18. [PMID: 34725311 DOI: 10.1088/1741-2552/ac2fc4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 10/14/2021] [Indexed: 11/11/2022]
Abstract
A brain-computer interface (BCI) aims to derive commands from the user's brain activity in order to relay them to an external device. To do so, it can either detect a spontaneous change in the mental state, in the so-called 'active' BCIs, or a transient or sustained change in the brain response to an external stimulation, in 'reactive' BCIs. In the latter, external stimuli are perceived by the user through a sensory channel, usually sight or hearing. When the stimulation is sustained and periodical, the brain response reaches an oscillatory steady-state that can be detected rather easily. We focus our attention on electroencephalography-based BCIs (EEG-based BCI) in which a periodical signal, either mechanical or electrical, stimulates the user skin. This type of stimulus elicits a steady-state response of the somatosensory system that can be detected in the recorded EEG. The oscillatory and phase-locked voltage component characterising this response is called a steady-state somatosensory-evoked potential (SSSEP). It has been shown that the amplitude of the SSSEP is modulated by specific mental tasks, for instance when the user focuses their attention or not to the somatosensory stimulation, allowing the translation of this variation into a command. Actually, SSSEP-based BCIs may benefit from straightforward analysis techniques of EEG signals, like reactive BCIs, while allowing self-paced interaction, like active BCIs. In this paper, we present a survey of scientific literature related to EEG-based BCI exploiting SSSEP. Firstly, we endeavour to describe the main characteristics of SSSEPs and the calibration techniques that allow the tuning of stimulation in order to maximise their amplitude. Secondly, we present the signal processing and data classification algorithms implemented by authors in order to elaborate commands in their SSSEP-based BCIs, as well as the classification performance that they evaluated on user experiments.
Collapse
Affiliation(s)
- Jimmy Petit
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| | - José Rouillard
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| | - François Cabestaing
- University of Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| |
Collapse
|
22
|
Verdonk C, Trousselard M, Di Bernardi Luft C, Medani T, Billaud JB, Ramdani C, Canini F, Claverie D, Jaumard-Hakoun A, Vialatte F. The heartbeat evoked potential does not support strong interoceptive sensibility in trait mindfulness. Psychophysiology 2021; 58:e13891. [PMID: 34227116 DOI: 10.1111/psyp.13891] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Revised: 05/07/2021] [Accepted: 06/10/2021] [Indexed: 12/27/2022]
Abstract
The enhancement of body awareness is proposed as one of the cognitive mechanisms that characterize mindfulness. To date, this hypothesis is supported by self-report and behavioral measures but still lacks physiological evidence. The current study investigated relation between trait mindfulness (i.e., individual differences in the ability to be mindful in daily life) and body awareness in combining a self-report measure (Multidimensional Assessment of Interoceptive Awareness [MAIA] questionnaire) with analysis of the heartbeat evoked potential (HEP), which is an event-related potential reflecting the cortical processing of the heartbeat. The HEP data were collected from 17 healthy participants under five minutes of resting-state condition. In addition, each participant completed the Freiburg Mindfulness Inventory and the MAIA questionnaire. Taking account of the important variability of HEP effects, analyses were replicated with the same participants three times (in three distinct sessions). First, group-level analyses showed that HEP amplitude and trait mindfulness do not correlate. Secondly, we observed that HEP amplitude could positively correlate with self-reported body awareness; however, this association was unreliable over time. Interestingly, we found that HEP measure shows very poor reliability over time at the individual level, potentially explaining the lack of reliable association between HEP and psychological traits. Lastly, a reliable positive correlation was found between self-reported trait mindfulness and body awareness. Taken together, these findings provide preliminary evidence that the HEP might not support the increased subjective body awareness in trait mindfulness, thus suggesting that perhaps objective and subjective measures of body awareness could be independent.
Collapse
Affiliation(s)
- Charles Verdonk
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France.,Brain Plasticity Unit, CNRS, ESPCI Paris, PSL University, Paris, France
| | - Marion Trousselard
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France.,French Military Health Service Academy, Paris, France
| | | | - Takfarinas Medani
- Brain Plasticity Unit, CNRS, ESPCI Paris, PSL University, Paris, France
| | - Jean-Baptiste Billaud
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France
| | - Céline Ramdani
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France
| | - Frédéric Canini
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France.,French Military Health Service Academy, Paris, France
| | - Damien Claverie
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France
| | | | - François Vialatte
- Brain Plasticity Unit, CNRS, ESPCI Paris, PSL University, Paris, France
| |
Collapse
|
23
|
Meng M, Yin X, She Q, Gao Y, Kong W, Luo Z. Sparse representation-based classification with two-dimensional dictionary optimization for motor imagery EEG pattern recognition. J Neurosci Methods 2021; 361:109274. [PMID: 34229027 DOI: 10.1016/j.jneumeth.2021.109274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2021] [Revised: 06/19/2021] [Accepted: 07/01/2021] [Indexed: 11/25/2022]
Abstract
BACKGROUND Sparse representation-based classification (SRC) has more advantages in motor imagery EEG pattern recognition, and the quality of dictionary construction directly determines the performance of SRC. In this paper, we proposed a two-dimensional dictionary optimization (TDDO) method to directly improve the performance of SRC. NEW METHOD Firstly, an initial dictionary was constructed with multi-band features extracted by filter band common spatial pattern (FBCSP). Then Lasso regression is used to select significant features in each atom synchronously in the horizontal direction, and the KNN-based method is used to clean up noise atoms in the vertical direction. Finally, an SRC method by training samples linearly representing test samples was implemented in classification. RESULTS The results show the necessity and rationality of TDDO-SRC method. The highest average classification accuracy of 86.5% and 92.4% is obtained on two public datasets. COMPARISON WITH EXISTING METHOD(S) The proposed method has more superior classification accuracy compared to traditional methods and existing winners' methods. CONCLUSIONS The quality of dictionary construction has a great impact on the robustness of SRC. And compared with the original SRC, the classification accuracy of the optimized TDDO-SRC is greatly improved.
Collapse
Affiliation(s)
- Ming Meng
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China; Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China,.
| | - Xu Yin
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
| | - Qingshan She
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China; Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Yunyuan Gao
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China; Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Wanzeng Kong
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou 310018, China
| | - Zhizeng Luo
- Institute of Intelligent Control and Robotics, Hangzhou Dianzi University, Hangzhou 310018, China
| |
Collapse
|
24
|
De Venuto D, Mezzina G. A Single-Trial P300 Detector Based on Symbolized EEG and Autoencoded-(1D)CNN to Improve ITR Performance in BCIs. SENSORS 2021; 21:s21123961. [PMID: 34201381 PMCID: PMC8226883 DOI: 10.3390/s21123961] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 06/02/2021] [Accepted: 06/07/2021] [Indexed: 12/01/2022]
Abstract
In this paper, we propose a breakthrough single-trial P300 detector that maximizes the information translate rate (ITR) of the brain–computer interface (BCI), keeping high recognition accuracy performance. The architecture, designed to improve the portability of the algorithm, demonstrated full implementability on a dedicated embedded platform. The proposed P300 detector is based on the combination of a novel pre-processing stage based on the EEG signals symbolization and an autoencoded convolutional neural network (CNN). The proposed system acquires data from only six EEG channels; thus, it treats them with a low-complexity preprocessing stage including baseline correction, windsorizing and symbolization. The symbolized EEG signals are then sent to an autoencoder model to emphasize those temporal features that can be meaningful for the following CNN stage. This latter consists of a seven-layer CNN, including a 1D convolutional layer and three dense ones. Two datasets have been analyzed to assess the algorithm performance: one from a P300 speller application in BCI competition III data and one from self-collected data during a fluid prototype car driving experiment. Experimental results on the P300 speller dataset showed that the proposed method achieves an average ITR (on two subjects) of 16.83 bits/min, outperforming by +5.75 bits/min the state-of-the-art for this parameter. Jointly with the speed increase, the recognition performance returned disruptive results in terms of the harmonic mean of precision and recall (F1-Score), which achieve 51.78 ± 6.24%. The same method used in the prototype car driving led to an ITR of ~33 bit/min with an F1-Score of 70.00% in a single-trial P300 detection context, allowing fluid usage of the BCI for driving purposes. The realized network has been validated on an STM32L4 microcontroller target, for complexity and implementation assessment. The implementation showed an overall resource occupation of 5.57% of the total available ROM, ~3% of the available RAM, requiring less than 3.5 ms to provide the classification outcome.
Collapse
|
25
|
Alhade BA, Ahmed IS, Kamil BZ. Brain Computer Interface using EEG Based Sequential Minimal Optimization algorithms. JOURNAL OF PHYSICS: CONFERENCE SERIES 2021; 1879:022092. [DOI: 10.1088/1742-6596/1879/2/022092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
Abstract
The concept of interfacing brains with robots/machines has been capturing human interests for a long time. The technology of the Brain-computer interface (BCI) has been aimed at building an interface between the brain and any electronic/electrical device (such as, smart home appliance, a wheelchair, and robotics devices) with the use of the electroencephalogram (EEG) that can be defined as a non-invasive approach for the measurement of the electrical potentials from the electrodes that have been placed on the scalp, produced by the activity of the brain. Over the past years, pattern classification was a highly challenging research field. Presently, the tasks of the pattern classification. In this paper, we chose motor imagery with the use of the single trial EEG signal, the SOM has been utilized to classify the signal processing algorithm ( FICA). In comparison to other algorithms of the EEG signal analyses. It has achieved a classification accuracy of up to 88.% in comparison with the other method where the reported accuracy has been 65%. The SOM classification algorithm has been fast, simple, efficient, and easy to use. It achieved satisfactory results at the BCI.
Collapse
|
26
|
Guger C, Prabhakaran V, Spataro R, Krusienski DJ, Hebb AO. Editorial: Breakthrough BCI Applications in Medicine. Front Neurosci 2021; 14:598247. [PMID: 33390884 PMCID: PMC7772388 DOI: 10.3389/fnins.2020.598247] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2020] [Accepted: 10/07/2020] [Indexed: 11/13/2022] Open
Affiliation(s)
| | - Vivek Prabhakaran
- Radiology Department, University of Wisconsin-Madison, Madison, WI, United States
| | - Rossella Spataro
- g.tec medical engineering GmbH, Schiedlberg, Austria.,University of Palermo, Palermo, Italy
| | - Dean J Krusienski
- Biomedical Engineering, Virginia Commonwealth University, Richmond, VA, United States
| | - Adam O Hebb
- Knoebel Institute for Healthy Aging, University of Denver, Denver, CO, United States
| |
Collapse
|
27
|
Kawala-Sterniuk A, Browarska N, Al-Bakri A, Pelc M, Zygarlicki J, Sidikova M, Martinek R, Gorzelanczyk EJ. Summary of over Fifty Years with Brain-Computer Interfaces-A Review. Brain Sci 2021; 11:43. [PMID: 33401571 PMCID: PMC7824107 DOI: 10.3390/brainsci11010043] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Revised: 12/25/2020] [Accepted: 12/27/2020] [Indexed: 11/16/2022] Open
Abstract
Over the last few decades, the Brain-Computer Interfaces have been gradually making their way to the epicenter of scientific interest. Many scientists from all around the world have contributed to the state of the art in this scientific domain by developing numerous tools and methods for brain signal acquisition and processing. Such a spectacular progress would not be achievable without accompanying technological development to equip the researchers with the proper devices providing what is absolutely necessary for any kind of discovery as the core of every analysis: the data reflecting the brain activity. The common effort has resulted in pushing the whole domain to the point where the communication between a human being and the external world through BCI interfaces is no longer science fiction but nowadays reality. In this work we present the most relevant aspects of the BCIs and all the milestones that have been made over nearly 50-year history of this research domain. We mention people who were pioneers in this area as well as we highlight all the technological and methodological advances that have transformed something available and understandable by a very few into something that has a potential to be a breathtaking change for so many. Aiming to fully understand how the human brain works is a very ambitious goal and it will surely take time to succeed. However, even that fraction of what has already been determined is sufficient e.g., to allow impaired people to regain control on their lives and significantly improve its quality. The more is discovered in this domain, the more benefit for all of us this can potentially bring.
Collapse
Affiliation(s)
- Aleksandra Kawala-Sterniuk
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Natalia Browarska
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Amir Al-Bakri
- Department of Biomedical Engineering, College of Engineering, University of Babylon, 51001 Babylon, Iraq;
| | - Mariusz Pelc
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
- Department of Computing and Information Systems, University of Greenwich, London SE10 9LS, UK
| | - Jaroslaw Zygarlicki
- Faculty of Electrical Engineering, Automatic Control and Informatics, Opole University of Technology, 45-758 Opole, Poland; (N.B.); (M.P.); (J.Z.)
| | - Michaela Sidikova
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Radek Martinek
- Department of Cybernetics and Biomedical Engineering, VSB-Technical University Ostrava—FEECS, 708 00 Ostrava-Poruba, Czech Republic; (M.S.); (R.M.)
| | - Edward Jacek Gorzelanczyk
- Department of Theoretical Basis of BioMedical Sciences and Medical Informatics, Nicolaus Copernicus University, Collegium Medicum, 85-067 Bydgoszcz, Poland;
- Institute of Philosophy, Kazimierz Wielki University, 85-092 Bydgoszcz, Poland
- Babinski Specialist Psychiatric Healthcare Center, Outpatient Addiction Treatment, 91-229 Lodz, Poland
- The Society for the Substitution Treatment of Addiction “Medically Assisted Recovery”, 85-791 Bydgoszcz, Poland
| |
Collapse
|
28
|
Fernández-Rodríguez Á, Ron-Angevin R, Sanz-Arigita EJ, Parize A, Esquirol J, Perrier A, Laur S, André JM, Lespinet-Najib V, Garcia L. Effect of Distracting Background Speech in an Auditory Brain-Computer Interface. Brain Sci 2021; 11:brainsci11010039. [PMID: 33401410 PMCID: PMC7823829 DOI: 10.3390/brainsci11010039] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2020] [Revised: 11/12/2020] [Accepted: 12/23/2020] [Indexed: 11/16/2022] Open
Abstract
Studies so far have analyzed the effect of distractor stimuli in different types of brain-computer interface (BCI). However, the effect of a background speech has not been studied using an auditory event-related potential (ERP-BCI), a convenient option when the visual path cannot be adopted by users. Thus, the aim of the present work is to examine the impact of a background speech on selection performance and user workload in auditory BCI systems. Eleven participants tested three conditions: (i) auditory BCI control condition, (ii) auditory BCI with a background speech to ignore (non-attentional condition), and (iii) auditory BCI while the user has to pay attention to the background speech (attentional condition). The results demonstrated that, despite no significant differences in performance, shared attention to auditory BCI and background speech required a higher cognitive workload. In addition, the P300 target stimuli in the non-attentional condition were significantly higher than those in the attentional condition for several channels. The non-attentional condition was the only condition that showed significant differences in the amplitude of the P300 between target and non-target stimuli. The present study indicates that background speech, especially when it is attended to, is an important interference that should be avoided while using an auditory BCI.
Collapse
Affiliation(s)
| | - Ricardo Ron-Angevin
- UMA-BCI Group, Departamento de Tecnología Electrónica, Universidad de Málaga, 29071 Malaga, Spain;
- Correspondence:
| | - Ernesto J. Sanz-Arigita
- Neuro and Aging and Human Cognition, INCIA-UMR 5287-CNRS, Université de Bordeaux, 33076 Bordeaux, France;
| | - Antoine Parize
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Juliette Esquirol
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Alban Perrier
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Simon Laur
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Jean-Marc André
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Véronique Lespinet-Najib
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| | - Liliana Garcia
- Laboratoire IMS, CNRS UMR 5218, Cognitive Team, Bordeaux INP-ENSC, 33400 Talence, France; (A.P.); (J.E.); (A.P.); (S.L.); (J.M.-A.); (V.L.-N.); (L.G.)
| |
Collapse
|
29
|
Kirasirova L, Bulanov V, Ossadtchi A, Kolsanov A, Pyatin V, Lebedev M. A P300 Brain-Computer Interface With a Reduced Visual Field. Front Neurosci 2020; 14:604629. [PMID: 33343290 PMCID: PMC7744588 DOI: 10.3389/fnins.2020.604629] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Accepted: 11/09/2020] [Indexed: 11/13/2022] Open
Abstract
A P300 brain-computer interface (BCI) is a paradigm, where text characters are decoded from event-related potentials (ERPs). In a popular implementation, called P300 speller, a subject looks at a display where characters are flashing and selects one character by attending to it. The selection is recognized as the item with the strongest ERP. The speller performs well when cortical responses to target and non-target stimuli are sufficiently different. Although many strategies have been proposed for improving the BCI spelling, a relatively simple one received insufficient attention in the literature: reduction of the visual field to diminish the contribution from non-target stimuli. Previously, this idea was implemented in a single-stimulus switch that issued an urgent command like stopping a robot. To tackle this approach further, we ran a pilot experiment where ten subjects operated a traditional P300 speller or wore a binocular aperture that confined their sight to the central visual field. As intended, visual field restriction resulted in a replacement of non-target ERPs with EEG rhythms asynchronous to stimulus periodicity. Changes in target ERPs were found in half of the subjects and were individually variable. While classification accuracy was slightly better for the aperture condition (84.3 ± 2.9%, mean ± standard error) than the no-aperture condition (81.0 ± 2.6%), this difference was not statistically significant for the entire sample of subjects (N = 10). For both the aperture and no-aperture conditions, classification accuracy improved over 4 days of training, more so for the aperture condition (from 72.0 ± 6.3% to 87.0 ± 3.9% and from 72.0 ± 5.6% to 97.0 ± 2.2% for the no-aperture and aperture conditions, respectively). Although in this study BCI performance was not substantially altered, we suggest that with further refinement this approach could speed up BCI operations and reduce user fatigue. Additionally, instead of wearing an aperture, non-targets could be removed algorithmically or with a hybrid interface that utilizes an eye tracker. We further discuss how a P300 speller could be improved by taking advantage of the different physiological properties of the central and peripheral vision. Finally, we suggest that the proposed experimental approach could be used in basic research on the mechanisms of visual processing.
Collapse
Affiliation(s)
| | - Vladimir Bulanov
- Laboratory of Mathematical Processing of Biological Information, IT Universe Ltd, Samara, Russia
| | - Alexei Ossadtchi
- Center for Bioelectric Interfaces of the Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| | | | | | - Mikhail Lebedev
- Center for Bioelectric Interfaces of the Institute for Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
- Department of Information and Internet Technologies of Digital Health Institute, I.M. Sechenov First Moscow State Medical University, Moscow, Russia
- Center For Neurobiology and Brain Restoration, Skolkovo Institute of Science and Technology, Moscow, Russia
| |
Collapse
|
30
|
Polich J. 50+ years of P300: Where are we now? Psychophysiology 2020; 57:e13616. [DOI: 10.1111/psyp.13616] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 05/05/2020] [Accepted: 05/12/2020] [Indexed: 11/30/2022]
Affiliation(s)
- John Polich
- The Scripps Research Institute La Jolla CA USA
| |
Collapse
|