1
|
Van Den Kerchove A, Si-Mohammed H, Van Hulle MM, Cabestaing F. Correcting for ERP latency jitter improves gaze-independent BCI decoding. J Neural Eng 2024; 21:046013. [PMID: 38959876 DOI: 10.1088/1741-2552/ad5ec0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Accepted: 07/03/2024] [Indexed: 07/05/2024]
Abstract
Objective.Patients suffering from heavy paralysis or Locked-in-Syndrome can regain communication using a Brain-Computer Interface (BCI). Visual event-related potential (ERP) based BCI paradigms exploit visuospatial attention (VSA) to targets laid out on a screen. However, performance drops if the user does not direct their eye gaze at the intended target, harming the utility of this class of BCIs for patients suffering from eye motor deficits. We aim to create an ERP decoder that is less dependent on eye gaze.Approach.ERP component latency jitter plays a role in covert visuospatial attention (VSA) decoding. We introduce a novel decoder which compensates for these latency effects, termed Woody Classifier-based Latency Estimation (WCBLE). We carried out a BCI experiment recording ERP data in overt and covert visuospatial attention (VSA), and introduce a novel special case of covert VSA termed split VSA, simulating the experience of patients with severely impaired eye motor control. We evaluate WCBLE on this dataset and the BNCI2014-009 dataset, within and across VSA conditions to study the dependency on eye gaze and the variation thereof during the experiment.Main results.WCBLE outperforms state-of-the-art methods in the VSA conditions of interest in gaze-independent decoding, without reducing overt VSA performance. Results from across-condition evaluation show that WCBLE is more robust to varying VSA conditions throughout a BCI operation session.Significance. Together, these results point towards a pathway to achieving gaze independence through suited ERP decoding. Our proposed gaze-independent solution enhances decoding performance in those cases where performing overt VSA is not possible.
Collapse
Affiliation(s)
- A Van Den Kerchove
- Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
- KU Leuven, Department of Neurosciences, Laboratory for Neuro- & Psychophysiology, Campus Gasthuisberg O&N2, Herestraat 49 bus 1021, BE-3000 Leuven, Belgium
| | - H Si-Mohammed
- Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| | - M M Van Hulle
- KU Leuven, Department of Neurosciences, Laboratory for Neuro- & Psychophysiology, Campus Gasthuisberg O&N2, Herestraat 49 bus 1021, BE-3000 Leuven, Belgium
| | - F Cabestaing
- Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France
| |
Collapse
|
2
|
Poststroke Cognitive Impairment Research Progress on Application of Brain-Computer Interface. BIOMED RESEARCH INTERNATIONAL 2022; 2022:9935192. [PMID: 35252458 PMCID: PMC8896931 DOI: 10.1155/2022/9935192] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 12/20/2021] [Accepted: 12/23/2021] [Indexed: 12/19/2022]
Abstract
Brain-computer interfaces (BCIs), a new type of rehabilitation technology, pick up nerve cell signals, identify and classify their activities, and convert them into computer-recognized instructions. This technique has been widely used in the rehabilitation of stroke patients in recent years and appears to promote motor function recovery after stroke. At present, the application of BCI in poststroke cognitive impairment is increasing, which is a common complication that also affects the rehabilitation process. This paper reviews the promise and potential drawbacks of using BCI to treat poststroke cognitive impairment, providing a solid theoretical basis for the application of BCI in this area.
Collapse
|
3
|
Stojic F, Chau T. Nonspecific Visuospatial Imagery as a Novel Mental Task for Online EEG-Based BCI Control. Int J Neural Syst 2020; 30:2050026. [PMID: 32498642 DOI: 10.1142/s0129065720500264] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Brain-computer interfaces (BCIs) can provide a means of communication to individuals with severe motor disorders, such as those presenting as locked-in. Many BCI paradigms rely on motor neural pathways, which are often impaired in these individuals. However, recent findings suggest that visuospatial function may remain intact. This study aimed to determine whether visuospatial imagery, a previously unexplored task, could be used to signify intent in an online electroencephalography (EEG)-based BCI. Eighteen typically developed participants imagined checkerboard arrow stimuli in four quadrants of the visual field in 5-s trials, while signals were collected using 16 dry electrodes over the visual cortex. In online blocks, participants received graded visual feedback based on their performance. An initial BCI pipeline (visuospatial imagery classifier I) attained a mean accuracy of [Formula: see text]% classifying rest against visuospatial imagery in online trials. This BCI pipeline was further improved using restriction to alpha band features (visuospatial imagery classifier II), resulting in a mean pseudo-online accuracy of [Formula: see text]%. Accuracies exceeded the threshold for practical BCIs in 12 participants. This study supports the use of visuospatial imagery as a real-time, binary EEG-BCI control paradigm.
Collapse
Affiliation(s)
- Filip Stojic
- Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Canada.,Institute of Biomaterials and Biomedical Engineering, University of Toronto, 27 King's College Circle, Toronto, Ontario, Canada M5S 1A1, Canada.,Terrance Donnelly Centre for Cellular and Biomolecular Research, 160 College St, Toronto, Ontario, Canada M5S 3E1, Canada
| | - Tom Chau
- Paediatric Rehabilitation Intelligent Systems, Multidisciplinary (PRISM) Laboratory, 150 Kilgour Rd, East York, Ontario, Canada M4G 1R8, Canada
| |
Collapse
|
4
|
Liu B, Huang X, Wang Y, Chen X, Gao X. BETA: A Large Benchmark Database Toward SSVEP-BCI Application. Front Neurosci 2020; 14:627. [PMID: 32655358 PMCID: PMC7324867 DOI: 10.3389/fnins.2020.00627] [Citation(s) in RCA: 55] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2020] [Accepted: 05/20/2020] [Indexed: 12/31/2022] Open
Abstract
The brain-computer interface (BCI) provides an alternative means to communicate and it has sparked growing interest in the past two decades. Specifically, for Steady-State Visual Evoked Potential (SSVEP) based BCI, marked improvement has been made in the frequency recognition method and data sharing. However, the number of pubic databases is still limited in this field. Therefore, we present a BEnchmark database Towards BCI Application (BETA) in the study. The BETA database is composed of 64-channel Electroencephalogram (EEG) data of 70 subjects performing a 40-target cued-spelling task. The design and the acquisition of the BETA are in pursuit of meeting the demand from real-world applications and it can be used as a test-bed for these scenarios. We validate the database by a series of analyses and conduct the classification analysis of eleven frequency recognition methods on BETA. We recommend using the metric of wide-band signal-to-noise ratio (SNR) and BCI quotient to characterize the SSVEP at the single-trial and population levels, respectively. The BETA database can be downloaded from the following link http://bci.med.tsinghua.edu.cn/download.html.
Collapse
Affiliation(s)
- Bingchuan Liu
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Xiaoshan Huang
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| | - Yijun Wang
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
| | - Xiaogang Chen
- Institute of Biomedical Engineering, Chinese Academy of Medical Sciences and Peking Union Medical College, Tianjin, China
| | - Xiaorong Gao
- Department of Biomedical Engineering, Tsinghua University, Beijing, China
| |
Collapse
|
5
|
Abstract
A brain–computer interface (BCI) has been extensively studied to develop a novel communication system for disabled people using their brain activities. An asynchronous BCI system is more realistic and practical than a synchronous BCI system, in that, BCI commands can be generated whenever the user wants. However, the relatively low performance of an asynchronous BCI system is problematic because redundant BCI commands are required to correct false-positive operations. To significantly reduce the number of false-positive operations of an asynchronous BCI system, a two-step approach has been proposed using a brain-switch that first determines whether the user wants to use an asynchronous BCI system before the operation of the asynchronous BCI system. This study presents a systematic review of the state-of-the-art brain-switch techniques and future research directions. To this end, we reviewed brain-switch research articles published from 2000 to 2019 in terms of their (a) neuroimaging modality, (b) paradigm, (c) operation algorithm, and (d) performance.
Collapse
|
6
|
Choi SI, Han CH, Choi GY, Shin J, Song KS, Im CH, Hwang HJ. On the Feasibility of Using an Ear-EEG to Develop an Endogenous Brain-Computer Interface. SENSORS (BASEL, SWITZERLAND) 2018; 18:E2856. [PMID: 30158505 PMCID: PMC6165202 DOI: 10.3390/s18092856] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/20/2018] [Revised: 08/26/2018] [Accepted: 08/28/2018] [Indexed: 11/27/2022]
Abstract
Brain-computer interface (BCI) studies based on electroencephalography (EEG) measured around the ears (ear-EEGs) have mostly used exogenous paradigms involving brain activity evoked by external stimuli. The objective of this study is to investigate the feasibility of ear-EEGs for development of an endogenous BCI system that uses self-modulated brain activity. We performed preliminary and main experiments where EEGs were measured on the scalp and behind the ears to check the reliability of ear-EEGs as compared to scalp-EEGs. In the preliminary and main experiments, subjects performed eyes-open and eyes-closed tasks, and they performed mental arithmetic (MA) and light cognitive (LC) tasks, respectively. For data analysis, the brain area was divided into four regions of interest (ROIs) (i.e., frontal, central, occipital, and ear area). The preliminary experiment showed that the degree of alpha activity increase of the ear area with eyes closed is comparable to those of other ROIs (occipital > ear > central > frontal). In the main experiment, similar event-related (de)synchronization (ERD/ERS) patterns were observed between the four ROIs during MA and LC, and all ROIs showed the mean classification accuracies above 70% required for effective binary communication (MA vs. LC) (occipital = ear = central = frontal). From the results, we demonstrated that ear-EEG can be used to develop an endogenous BCI system based on cognitive tasks without external stimuli, which allows the usability of ear-EEGs to be extended.
Collapse
Affiliation(s)
- Soo-In Choi
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea.
| | - Chang-Hee Han
- Berlin Institute of Technology, Machine Learning Group, Marchstrasse 23, 10587 Berlin, Germany.
| | - Ga-Young Choi
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea.
| | - Jaeyoung Shin
- Department of Biomedical Engineering, Hanyang University, Seoul 04763, Korea.
| | - Kwang Soup Song
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea.
| | - Chang-Hwan Im
- Department of Biomedical Engineering, Hanyang University, Seoul 04763, Korea.
| | - Han-Jeong Hwang
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gumi 39177, Korea.
| |
Collapse
|
7
|
Eyes-closed hybrid brain-computer interface employing frontal brain activation. PLoS One 2018; 13:e0196359. [PMID: 29734383 PMCID: PMC5937739 DOI: 10.1371/journal.pone.0196359] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2017] [Accepted: 04/11/2018] [Indexed: 11/23/2022] Open
Abstract
Brain-computer interfaces (BCIs) have been studied extensively in order to establish a non-muscular communication channel mainly for patients with impaired motor functions. However, many limitations remain for BCIs in clinical use. In this study, we propose a hybrid BCI that is based on only frontal brain areas and can be operated in an eyes-closed state for end users with impaired motor and declining visual functions. In our experiment, electroencephalography (EEG) and near-infrared spectroscopy (NIRS) were simultaneously measured while 12 participants performed mental arithmetic (MA) and remained relaxed (baseline state: BL). To evaluate the feasibility of the hybrid BCI, we classified MA- from BL-related brain activation. We then compared classification accuracies using two unimodal BCIs (EEG and NIRS) and the hybrid BCI in an offline mode. The classification accuracy of the hybrid BCI (83.9 ± 10.3%) was shown to be significantly higher than those of unimodal EEG-based (77.3 ± 15.9%) and NIRS-based BCI (75.9 ± 6.3%). The analytical results confirmed performance improvement with the hybrid BCI, particularly for only frontal brain areas. Our study shows that an eyes-closed hybrid BCI approach based on frontal areas could be applied to neurodegenerative patients who lost their motor functions, including oculomotor functions.
Collapse
|
8
|
Treder MS. Improving SNR and Reducing Training Time of Classifiers in Large Datasets via Kernel Averaging. Brain Inform 2018. [DOI: 10.1007/978-3-030-05587-5_23] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
|
9
|
Won DO, Hwang HJ, Kim DM, Muller KR, Lee SW. Motion-Based Rapid Serial Visual Presentation for Gaze-Independent Brain-Computer Interfaces. IEEE Trans Neural Syst Rehabil Eng 2017; 26:334-343. [PMID: 28809703 DOI: 10.1109/tnsre.2017.2736600] [Citation(s) in RCA: 61] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Most event-related potential (ERP)-based brain-computer interface (BCI) spellers primarily use matrix layouts and generally require moderate eye movement for successful operation. The fundamental objective of this paper is to enhance the perceptibility of target characters by introducing motion stimuli to classical rapid serial visual presentation (RSVP) spellers that do not require any eye movement, thereby applying them to paralyzed patients with oculomotor dysfunctions. To test the feasibility of the proposed motion-based RSVP paradigm, we implemented three RSVP spellers: 1) fixed-direction motion (FM-RSVP); 2) random-direction motion (RM-RSVP); and 3) (the conventional) non-motion stimulation (NM-RSVP), and evaluated the effect of the three different stimulation methods on spelling performance. The two motion-based stimulation methods, FM- and RM-RSVP, showed shorter P300 latency and higher P300 amplitudes (i.e., 360.4-379.6 ms; 5.5867- ) than the NM-RSVP (i.e., 480.4 ms; ). This led to higher and more stable performances for FM- and RM-RSVP spellers than NM-RSVP speller (i.e., 79.06±6.45% for NM-RSVP, 90.60±2.98% for RM-RSVP, and 92.74±2.55% for FM-RSVP). In particular, the proposed motion-based RSVP paradigm was significantly beneficial for about half of the subjects who might not accurately perceive rapidly presented static stimuli. These results indicate that the use of proposed motion-based RSVP paradigm is more beneficial for target recognition when developing BCI applications for severely paralyzed patients with complex ocular dysfunctions.
Collapse
|
10
|
Cecotti H, Ries AJ. Best practice for single-trial detection of event-related potentials: Application to brain-computer interfaces. Int J Psychophysiol 2017; 111:156-169. [DOI: 10.1016/j.ijpsycho.2016.07.500] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2015] [Revised: 07/02/2016] [Accepted: 07/16/2016] [Indexed: 11/30/2022]
|
11
|
Shin J, Müller KR, Hwang HJ. Near-infrared spectroscopy (NIRS)-based eyes-closed brain-computer interface (BCI) using prefrontal cortex activation due to mental arithmetic. Sci Rep 2016; 6:36203. [PMID: 27824089 PMCID: PMC5099935 DOI: 10.1038/srep36203] [Citation(s) in RCA: 41] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2016] [Accepted: 10/12/2016] [Indexed: 11/11/2022] Open
Abstract
We propose a near-infrared spectroscopy (NIRS)-based brain-computer interface (BCI) that can be operated in eyes-closed (EC) state. To evaluate the feasibility of NIRS-based EC BCIs, we compared the performance of an eye-open (EO) BCI paradigm and an EC BCI paradigm with respect to hemodynamic response and classification accuracy. To this end, subjects performed either mental arithmetic or imagined vocalization of the English alphabet as a baseline task with very low cognitive loading. The performances of two linear classifiers were compared; resulting in an advantage of shrinkage linear discriminant analysis (LDA). The classification accuracy of EC paradigm (75.6 ± 7.3%) was observed to be lower than that of EO paradigm (77.0 ± 9.2%), which was statistically insignificant (p = 0.5698). Subjects reported they felt it more comfortable (p = 0.057) and easier (p < 0.05) to perform the EC BCI tasks. The different task difficulty may become a cause of the slightly lower classification accuracy of EC data. From the analysis results, we could confirm the feasibility of NIRS-based EC BCIs, which can be a BCI option that may ultimately be of use for patients who cannot keep their eyes open consistently.
Collapse
Affiliation(s)
- Jaeyoung Shin
- Machine Learning Group, Berlin Institute of Technology (TU Berlin), Marchstr. 23, 10587 Berlin, Germany
| | - Klaus-R Müller
- Machine Learning Group, Berlin Institute of Technology (TU Berlin), Marchstr. 23, 10587 Berlin, Germany
- Department of Brain and Cognitive Engineering, Korea University, 136-713 Seoul, Korea
| | - Han-Jeong Hwang
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, 730-701 Gumi, Korea
| |
Collapse
|
12
|
Halder S, Takano K, Ora H, Onishi A, Utsumi K, Kansaku K. An Evaluation of Training with an Auditory P300 Brain-Computer Interface for the Japanese Hiragana Syllabary. Front Neurosci 2016; 10:446. [PMID: 27746716 PMCID: PMC5043244 DOI: 10.3389/fnins.2016.00446] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2016] [Accepted: 09/16/2016] [Indexed: 12/03/2022] Open
Abstract
Gaze-independent brain-computer interfaces (BCIs) are a possible communication channel for persons with paralysis. We investigated if it is possible to use auditory stimuli to create a BCI for the Japanese Hiragana syllabary, which has 46 Hiragana characters. Additionally, we investigated if training has an effect on accuracy despite the high amount of different stimuli involved. Able-bodied participants (N = 6) were asked to select 25 syllables (out of fifty possible choices) using a two step procedure: First the consonant (ten choices) and then the vowel (five choices). This was repeated on 3 separate days. Additionally, a person with spinal cord injury (SCI) participated in the experiment. Four out of six healthy participants reached Hiragana syllable accuracies above 70% and the information transfer rate increased from 1.7 bits/min in the first session to 3.2 bits/min in the third session. The accuracy of the participant with SCI increased from 12% (0.2 bits/min) to 56% (2 bits/min) in session three. Reliable selections from a 10 × 5 matrix using auditory stimuli were possible and performance is increased by training. We were able to show that auditory P300 BCIs can be used for communication with up to fifty symbols. This enables the use of the technology of auditory P300 BCIs with a variety of applications.
Collapse
Affiliation(s)
- Sebastian Halder
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
- Department of Psychology I, Institute of Psychology, University of WürzburgWürzburg, Germany
| | - Kouji Takano
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
| | - Hiroki Ora
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
- Brain Science Inspired Life Support Research Center, University of Electro-CommunicationsChofu, Japan
| | - Akinari Onishi
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
| | - Kota Utsumi
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
- Department of Neurology, Brain Research Institute, Niigata UniversityNiigata, Japan
| | - Kenji Kansaku
- Systems Neuroscience Section, Department of Rehabilitation for Brain Functions, Research Institute of National Rehabilitation Center for Persons with DisabilitiesTokorozawa, Japan
- Brain Science Inspired Life Support Research Center, University of Electro-CommunicationsChofu, Japan
| |
Collapse
|
13
|
Abstract
Brain-computer interfaces are systems that use signals recorded from the brain to enable communication and control applications for individuals who have impaired function. This technology has developed to the point that it is now being used by individuals who can actually benefit from it. However, there are several outstanding issues that prevent widespread use. These include the ease of obtaining high-quality recordings by home users, the speed, and accuracy of current devices and adapting applications to the needs of the user. In this chapter, we discuss some of these unsolved issues.
Collapse
|