1
|
Si Y, Wang Z, Xu G, Wang Z, Xu T, Zhou T, Hu H. Group-member selection for RSVP-based collaborative brain-computer interfaces. Front Neurosci 2024; 18:1402154. [PMID: 39234182 PMCID: PMC11371794 DOI: 10.3389/fnins.2024.1402154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Accepted: 07/30/2024] [Indexed: 09/06/2024] Open
Abstract
Objective The brain-computer interface (BCI) systems based on rapid serial visual presentation (RSVP) have been widely utilized for the detection of target and non-target images. Collaborative brain-computer interface (cBCI) effectively fuses electroencephalogram (EEG) data from multiple users to overcome the limitations of low single-user performance in single-trial event-related potential (ERP) detection in RSVP-based BCI systems. In a multi-user cBCI system, a superior group mode may lead to better collaborative performance and lower system cost. However, the key factors that enhance the collaboration capabilities of multiple users and how to further use these factors to optimize group mode remain unclear. Approach This study proposed a group-member selection strategy to optimize the group mode and improve the system performance for RSVP-based cBCI. In contrast to the conventional grouping of collaborators at random, the group-member selection strategy enabled pairing each user with a better collaborator and allowed tasks to be done with fewer collaborators. Initially, we introduced the maximum individual capability and maximum collaborative capability (MIMC) to select optimal pairs, improving the system classification performance. The sequential forward floating selection (SFFS) combined with MIMC then selected a sub-group, aiming to reduce the hardware and labor expenses in the cBCI system. Moreover, the hierarchical discriminant component analysis (HDCA) was used as a classifier for within-session conditions, and the Euclidean space data alignment (EA) was used to overcome the problem of inter-trial variability for cross-session analysis. Main results In this paper, we verified the effectiveness of the proposed group-member selection strategy on a public RSVP-based cBCI dataset. For the two-user matching task, the proposed MIMC had a significantly higher AUC and TPR and lower FPR than the common random grouping mode and the potential group-member selection method. Moreover, the SFFS with MIMC enabled a trade-off between maintaining performance and reducing the number of system users. Significance The results showed that our proposed MIMC effectively optimized the group mode, enhanced the classification performance in the two-user matching task, and could reduce the redundant information by selecting the sub-group in the RSVP-based multi-user cBCI systems.
Collapse
Affiliation(s)
- Yuan Si
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Zhenyu Wang
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
| | - Guiying Xu
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Zikai Wang
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Tianheng Xu
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- Shanghai Frontier Innovation Research Institute, Shanghai, China
| | - Ting Zhou
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- Shanghai Frontier Innovation Research Institute, Shanghai, China
- School of Microelectronics, Shanghai University, Shanghai, China
| | - Honglin Hu
- Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China
- University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
2
|
Lin S, Jiang J, Huang K, Li L, He X, Du P, Wu Y, Liu J, Li X, Huang Z, Zhou Z, Yu Y, Gao J, Lei M, Wu H. Advanced Electrode Technologies for Noninvasive Brain-Computer Interfaces. ACS NANO 2023; 17:24487-24513. [PMID: 38064282 DOI: 10.1021/acsnano.3c06781] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/27/2023]
Abstract
Brain-computer interfaces (BCIs) have garnered significant attention in recent years due to their potential applications in medical, assistive, and communication technologies. Building on this, noninvasive BCIs stand out as they provide a safe and user-friendly method for interacting with the human brain. In this work, we provide a comprehensive overview of the latest developments and advancements in material, design, and application of noninvasive BCIs electrode technology. We also explore the challenges and limitations currently faced by noninvasive BCI electrode technology and sketch out the technological roadmap from three dimensions: Materials and Design; Performances; Mode and Function. We aim to unite research efforts within the field of noninvasive BCI electrode technology, focusing on the consolidation of shared goals and fostering integrated development strategies among a diverse array of multidisciplinary researchers.
Collapse
Affiliation(s)
- Sen Lin
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Jingjing Jiang
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Kai Huang
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
- State Key Laboratory of Information Photonics and Optical Communications and School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
| | - Lei Li
- National Engineering Research Center of Electric Vehicles, Beijing Institute of Technology, Beijing 100081, China
| | - Xian He
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
| | - Peng Du
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
| | - Yufeng Wu
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
| | - Junchen Liu
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
- State Key Laboratory of Information Photonics and Optical Communications and School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
| | - Xilin Li
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
- Advanced Institute for Brain and Intelligence, Guangxi University, Nanning 530004, China
| | - Zhibao Huang
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Zenan Zhou
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Yuanhang Yu
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Jiaxin Gao
- School of Physical Science and Technology, Guangxi University, Nanning 530004, China
| | - Ming Lei
- State Key Laboratory of Information Photonics and Optical Communications and School of Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
| | - Hui Wu
- State Key Laboratory of New Ceramics and Fine Processing, School of Materials Science and Engineering, Tsinghua University, Beijing 100084, China
| |
Collapse
|
3
|
Amini Gougeh R, Falk TH. Enhancing motor imagery detection efficacy using multisensory virtual reality priming. FRONTIERS IN NEUROERGONOMICS 2023; 4:1080200. [PMID: 38236517 PMCID: PMC10790854 DOI: 10.3389/fnrgo.2023.1080200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 03/23/2023] [Indexed: 01/19/2024]
Abstract
Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent directly from electroencephalography (EEG) signals. For some users, however, it is difficult to elicit patterns in the EEG signal that can be detected with existing features and classifiers. As such, new user control strategies and training paradigms have been highly sought-after to help improve motor imagery performance. Virtual reality (VR) has emerged as one potential tool where improvements in user engagement and level of immersion have shown to improve BCI accuracy. Motor priming in VR, in turn, has shown to further enhance BCI accuracy. In this pilot study, we take the first steps to explore if multisensory VR motor priming, where haptic and olfactory stimuli are present, can improve motor imagery detection efficacy in terms of both improved accuracy and faster detection. Experiments with 10 participants equipped with a biosensor-embedded VR headset, an off-the-shelf scent diffusion device, and a haptic glove with force feedback showed that significant improvements in motor imagery detection could be achieved. Increased activity in the six common spatial pattern filters used were also observed and peak accuracy could be achieved with analysis windows that were 2 s shorter. Combined, the results suggest that multisensory motor priming prior to motor imagery could improve detection efficacy.
Collapse
Affiliation(s)
| | - Tiago H. Falk
- Institut National de la Recherche Scientifique-Energy, Materials and Telecommunications Center, University of Québec, Montreal, QC, Canada
| |
Collapse
|
4
|
Sommer W, Stapor K, Kończak G, Kotowski K, Fabian P, Ochab J, Bereś A, Ślusarczyk G. Changepoint Detection in Noisy Data Using a Novel Residuals Permutation-Based Method (RESPERM): Benchmarking and Application to Single Trial ERPs. Brain Sci 2022; 12:brainsci12050525. [PMID: 35624912 PMCID: PMC9139177 DOI: 10.3390/brainsci12050525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Revised: 04/13/2022] [Accepted: 04/19/2022] [Indexed: 11/30/2022] Open
Abstract
An important problem in many fields dealing with noisy time series, such as psychophysiological single trial data during learning or monitoring treatment effects over time, is detecting a change in the model underlying a time series. Here, we present a new method for detecting a single changepoint in a linear time series regression model, termed residuals permutation-based method (RESPERM). The optimal changepoint in RESPERM maximizes Cohen’s effect size with the parameters estimated by the permutation of residuals in a linear model. RESPERM was compared with the SEGMENTED method, a well-established and recommended method for detecting changepoints, using extensive simulated data sets, varying the amount and distribution characteristics of noise and the location of the change point. In time series with medium to large amounts of noise, the variance of the detected changepoint was consistently smaller for RESPERM than SEGMENTED. Finally, both methods were applied to a sample dataset of single trial amplitudes of the N250 ERP component during face learning. In conclusion, RESPERM appears to be well suited for changepoint detection especially in noisy data, making it the method of choice in neuroscience, medicine and many other fields.
Collapse
Affiliation(s)
- Werner Sommer
- Department of Psychology, Humboldt-University of Berlin, 10099 Berlin, Germany;
- Department of Psychology, Zhejiang Normal University, Jinhua 321000, China
| | - Katarzyna Stapor
- Department of Applied Informatics, Silesian University of Technology, 44-100 Gliwice, Poland; (K.K.); (P.F.)
- Correspondence:
| | - Grzegorz Kończak
- Department of Statistics, Econometrics and Mathematics, University of Economics in Katowice, 40-287 Katowice, Poland;
| | - Krzysztof Kotowski
- Department of Applied Informatics, Silesian University of Technology, 44-100 Gliwice, Poland; (K.K.); (P.F.)
| | - Piotr Fabian
- Department of Applied Informatics, Silesian University of Technology, 44-100 Gliwice, Poland; (K.K.); (P.F.)
| | - Jeremi Ochab
- Department of Theory of Complex Systems, Jagiellonian University, 30-348 Krakow, Poland;
| | - Anna Bereś
- Department of Cognitive Neuroscience and Neuroergonomics, Jagiellonian University, 30-348 Krakow, Poland;
| | - Grażyna Ślusarczyk
- Department of Design and Computer Graphics, Jagiellonian University, 30-348 Krakow, Poland;
| |
Collapse
|
5
|
A Collaborative Brain-Computer Interface Framework for Enhancing Group Detection Performance of Dynamic Visual Targets. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:4752450. [PMID: 35087580 PMCID: PMC8789438 DOI: 10.1155/2022/4752450] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Revised: 12/12/2021] [Accepted: 12/14/2021] [Indexed: 11/25/2022]
Abstract
The superiority of collaborative brain-computer interface (cBCI) in performance enhancement makes it an effective way to break through the performance bottleneck of the BCI-based dynamic visual target detection. However, the existing cBCIs focus on multi-mind information fusion with a static and unidirectional mode, lacking the information interaction and learning guidance among multiple agents. Here, we propose a novel cBCI framework to enhance the group detection performance of dynamic visual targets. Specifically, a mutual learning domain adaptation network (MLDANet) with information interaction, dynamic learning, and individual transferring abilities is developed as the core of the cBCI framework. MLDANet takes P3-sSDA network as individual network unit, introduces mutual learning strategy, and establishes a dynamic interactive learning mechanism between individual networks and collaborative decision-making at the neural decision level. The results indicate that the proposed MLDANet-cBCI framework can achieve the best group detection performance, and the mutual learning strategy can improve the detection ability of individual networks. In MLDANet-cBCI, the F1 scores of collaborative detection and individual network are 0.12 and 0.19 higher than those in the multi-classifier cBCI, respectively, when three minds collaborate. Thus, the proposed framework breaks through the traditional multi-mind collaborative mode and exhibits a superior group detection performance of dynamic visual targets, which is also of great significance for the practical application of multi-mind collaboration.
Collapse
|
6
|
Zheng L, Sun S, Zhao H, Pei W, Chen H, Gao X, Zhang L, Wang Y. A Cross-Session Dataset for Collaborative Brain-Computer Interfaces Based on Rapid Serial Visual Presentation. Front Neurosci 2020; 14:579469. [PMID: 33192265 PMCID: PMC7642747 DOI: 10.3389/fnins.2020.579469] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2020] [Accepted: 09/22/2020] [Indexed: 11/20/2022] Open
Abstract
Brain-computer interfaces (BCIs) based on rapid serial visual presentation (RSVP) have been widely used to categorize target and non-target images. However, it is still a challenge to detect single-trial event related potentials (ERPs) from electroencephalography (EEG) signals. Besides, the variability of EEG signal over time may cause difficulties of calibration in long-term system use. Recently, collaborative BCIs have been proposed to improve the overall BCI performance by fusing brain activities acquired from multiple subjects. For both individual and collaborative BCIs, feature extraction and classification algorithms that can be transferred across sessions can significantly facilitate system calibration. Although open datasets are highly efficient for developing algorithms, currently there is still a lack of datasets for a collaborative RSVP-based BCI. This paper presents a cross-session EEG dataset of a collaborative RSVP-based BCI system from 14 subjects, who were divided into seven groups. In collaborative BCI experiments, two subjects did the same target image detection tasks synchronously. All subjects participated in the same experiment twice with an average interval of ∼23 days. The results in data evaluation indicate that adequate signal processing algorithms can greatly enhance the cross-session BCI performance in both individual and collaborative conditions. Besides, compared with individual BCIs, the collaborative methods that fuse information from multiple subjects obtain significantly improved BCI performance. This dataset can be used for developing more efficient algorithms to enhance performance and practicality of a collaborative RSVP-based BCI system.
Collapse
Affiliation(s)
- Li Zheng
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China.,School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Sen Sun
- Department of Control Engineering, School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China
| | - Hongze Zhao
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
| | - Weihua Pei
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China.,School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| | - Hongda Chen
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
| | - Xiaorong Gao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Lijian Zhang
- Beijing Machine and Equipment Institute, Beijing, China
| | - Yijun Wang
- State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China.,School of Future Technology, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
7
|
Cinel C, Valeriani D, Poli R. Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects. Front Hum Neurosci 2019; 13:13. [PMID: 30766483 PMCID: PMC6365771 DOI: 10.3389/fnhum.2019.00013] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2018] [Accepted: 01/10/2019] [Indexed: 01/10/2023] Open
Abstract
Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.
Collapse
Affiliation(s)
- Caterina Cinel
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Davide Valeriani
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
- Department of Otolaryngology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, United States
| | - Riccardo Poli
- Brain Computer Interfaces and Neural Engineering Laboratory, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| |
Collapse
|
8
|
Matran-Fernandez A, Poli R. Towards the automated localisation of targets in rapid image-sifting by collaborative brain-computer interfaces. PLoS One 2017; 12:e0178498. [PMID: 28562664 PMCID: PMC5451058 DOI: 10.1371/journal.pone.0178498] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2017] [Accepted: 05/14/2017] [Indexed: 11/18/2022] Open
Abstract
The N2pc is a lateralised Event-Related Potential (ERP) that signals a shift of attention towards the location of a potential object of interest. We propose a single-trial target-localisation collaborative Brain-Computer Interface (cBCI) that exploits this ERP to automatically approximate the horizontal position of targets in aerial images. Images were presented by means of the rapid serial visual presentation technique at rates of 5, 6 and 10 Hz. We created three different cBCIs and tested a participant selection method in which groups are formed according to the similarity of participants' performance. The N2pc that is elicited in our experiments contains information about the position of the target along the horizontal axis. Moreover, combining information from multiple participants provides absolute median improvements in the area under the receiver operating characteristic curve of up to 21% (for groups of size 3) with respect to single-user BCIs. These improvements are bigger when groups are formed by participants with similar individual performance, and much of this effect can be explained using simple theoretical models. Our results suggest that BCIs for automated triaging can be improved by integrating two classification systems: one devoted to target detection and another to detect the attentional shifts associated with lateral targets.
Collapse
Affiliation(s)
- Ana Matran-Fernandez
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, Essex, United Kingdom
- * E-mail:
| | - Riccardo Poli
- School of Computer Science and Electronic Engineering, University of Essex, Colchester, Essex, United Kingdom
| |
Collapse
|
9
|
Cecotti H. Toward shift invariant detection of event-related potentials in non-invasive brain-computer interface. Pattern Recognit Lett 2015. [DOI: 10.1016/j.patrec.2015.01.015] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|