1
|
Tu Y, Li X, Lu ZL, Wang Y. Adaptive smoothing of retinotopic maps based on Teichmüller parametrization. Med Image Anal 2024; 93:103074. [PMID: 38160658 DOI: 10.1016/j.media.2023.103074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2022] [Revised: 12/20/2023] [Accepted: 12/21/2023] [Indexed: 01/03/2024]
Abstract
Retinotopic mapping, the mapping between visual inputs on the retina and neural responses on the cortical surface, is one of the fundamental topics in visual neuroscience. In human studies, retinotopic maps are conventionally constructed and processed by decoding blood oxygenation-level dependent (BOLD) functional magnetic resonance imaging (fMRI) responses to designed visual stimuli on the cortical surface. However, these methods frequently generate retinotopic maps that do not preserve topology, contradicting a fundamental property of retinotopic maps observed in neurophysiology. To address this problem, we propose an integrated approach to simultaneously refine the flattening from the 3D cortical surface to the 2D parametric space and adaptively smooth retinotopic perception centers in the visual space to make the retinotopic maps topological. One key element of the approach is the enhanced error tolerant Teichmüller mapping, which refines the parametrization by minimizing angle distortions and maximizing alignment to noisy landmarks. We validated our overall approach with synthetic and real retinotopic mapping datasets and applied it to compute cortical magnification factor (CMF). The results showed that the proposed approach was superior to other conventional retinotopic mapping methods in predicting BOLD fMRI time series and preserving topology.
Collapse
Affiliation(s)
- Yanshuai Tu
- School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA
| | - Xin Li
- School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA
| | - Zhong-Lin Lu
- Division of Arts and Sciences, New York University Shanghai, Shanghai, China; Center for Neural Science and Department of Psychology, New York University, New York, NY, USA; NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, China.
| | - Yalin Wang
- School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA.
| |
Collapse
|
2
|
Xiao X, Gao R, Zhou X, Yi W, Xu F, Wang K, Xu M, Ming D. A novel visual brain-computer interfaces paradigm based on evoked related potentials evoked by weak and small number of stimuli. Front Neurosci 2023; 17:1178283. [PMID: 37342465 PMCID: PMC10278229 DOI: 10.3389/fnins.2023.1178283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 03/20/2023] [Indexed: 06/23/2023] Open
Abstract
Introduction Traditional visual Brain-Computer Interfaces (v-BCIs) usually use large-size stimuli to attract more attention from users and then elicit more distinct and robust EEG responses, which would cause visual fatigue and limit the length of use of the system. On the contrary, small-size stimuli always need multiple and repeated stimulus to code more instructions and increase separability among each code. These common v-BCIs paradigms can cause problems such as redundant coding, long calibration time, and visual fatigue. Methods To address these problems, this study presented a novel v-BCI paradigm using weak and small number of stimuli, and realized a nine-instruction v-BCI system that controlled by only three tiny stimuli. Each of these stimuli were located between instructions, occupied area with eccentricities subtended 0.4°, and flashed in the row-column paradigm. The weak stimuli around each instruction would evoke specific evoked related potentials (ERPs), and a template-matching method based on discriminative spatial pattern (DSP) was employed to recognize these ERPs containing the intention of users. Nine subjects participated in the offline and online experiments using this novel paradigm. Results The average accuracy of the offline experiment was 93.46% and the online average information transfer rate (ITR) was 120.95 bits/min. Notably, the highest online ITR achieved 177.5 bits/min. Discussion These results demonstrate the feasibility of using a weak and small number of stimuli to implement a friendly v-BCI. Furthermore, the proposed novel paradigm achieved higher ITR than traditional ones using ERPs as the controlled signal, which showed its superior performance and may have great potential of being widely used in various fields.
Collapse
Affiliation(s)
- Xiaolin Xiao
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Runyuan Gao
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Xiaoyu Zhou
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
| | - Weibo Yi
- Beijing Institute of Mechanical Equipment, Beijing, China
| | - Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology, Shandong Academy of Sciences, Jinan, China
| | - Kun Wang
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| | - Minpeng Xu
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
- International School for Optoelectronic Engineering, Qilu University of Technology, Shandong Academy of Sciences, Jinan, China
| | - Dong Ming
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
| |
Collapse
|
3
|
Li R, Hu H, Zhao X, Wang Z, Xu G. A static paradigm based on illusion-induced VEP for brain-computer interfaces. J Neural Eng 2023; 20:026006. [PMID: 36808912 DOI: 10.1088/1741-2552/acbdc0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2023]
Abstract
OBJECTIVE Visual evoked potentials (VEPs) have been commonly applied in brain-computer interfaces (BCIs) due to their satisfactory classification performance recently. However, most existing methods with flickering or oscillating stimuli will induce visual fatigue under long-term training, thus restricting the implementation of VEP-based BCIs. To address this issue, a novel paradigm adopting static motion illusion based on illusion-induced visual evoked potential (IVEP) is proposed for BCIs to enhance visual experience and practicality. APPROACH This study explored the responses to baseline and illusion tasks including the Rotating-Tilted-Lines (RTL) illusion and Rotating-Snakes (RS) illusion. The distinguishable features were examined between different illusions by analyzing the event-related potentials (ERPs) and amplitude modulation of evoked oscillatory responses. MAIN RESULTS The illusion stimuli elicited VEPs in an early time window encompassing a negative component (N1) from 110 to 200 ms and a positive component (P2) between 210 and 300 ms. Based on the feature analysis, a filter bank was designed to extract discriminative signals. The task-related component analysis (TRCA) was used to evaluate the binary classification task performance of the proposed method. Then the highest accuracy of 86.67% was achieved with a data length of 0.6 s. SIGNIFICANCE The results of this study demonstrate that the static motion illusion paradigm has the feasibility of implementation and is promising for VEP-based BCI applications.
Collapse
Affiliation(s)
- Ruxue Li
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Honglin Hu
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Xi Zhao
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Zhenyu Wang
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute Chinese Academy of Sciences, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, 201210, CHINA
| | - Guiying Xu
- Intelligent Information and Communication Technology Research and Development Center, Shanghai Advanced Research Institute, 99 Haike Road, Pudong New Area, Shanghai, Shanghai, Shanghai, 201210, CHINA
| |
Collapse
|
4
|
Han J, Xu M, Xiao X, Yi W, Jung TP, Ming D. A high-speed hybrid brain-computer interface with more than 200 targets. J Neural Eng 2023; 20:016025. [PMID: 36608342 DOI: 10.1088/1741-2552/acb105] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 01/06/2023] [Indexed: 01/07/2023]
Abstract
Objective. Brain-computer interfaces (BCIs) have recently made significant strides in expanding their instruction set, which has attracted wide attention from researchers. The number of targets and commands is a key indicator of how well BCIs can decode the brain's intentions. No studies have reported a BCI system with over 200 targets.Approach. This study developed the first high-speed BCI system with up to 216 targets that were encoded by a combination of electroencephalography features, including P300, motion visual evoked potential (mVEP), and steady-state visual evoked potential (SSVEP). Specifically, the hybrid BCI paradigm used the time-frequency division multiple access strategy to elaborately tag targets with P300 and mVEP of different time windows, along with SSVEP of different frequencies. The hybrid features were then decoded by task-discriminant component analysis and linear discriminant analysis. Ten subjects participated in the offline and online cued-guided spelling experiments. Other ten subjects took part in online free-spelling experiments.Main results.The offline results showed that the mVEP and P300 components were prominent in the central, parietal, and occipital regions, while the most distinct SSVEP feature was in the occipital region. The online cued-guided spelling and free-spelling results showed that the proposed BCI system achieved an average accuracy of 85.37% ± 7.49% and 86.00% ± 5.98% for the 216-target classification, resulting in an average information transfer rate (ITR) of 302.83 ± 39.20 bits min-1and 204.47 ± 37.56 bits min-1, respectively. Notably, the peak ITR could reach up to 367.83 bits min-1.Significance.This study developed the first high-speed BCI system with more than 200 targets, which holds promise for extending BCI's application scenarios.
Collapse
Affiliation(s)
- Jin Han
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
| | - Minpeng Xu
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| | - Xiaolin Xiao
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| | - Weibo Yi
- Beijing Machine and Equipment Institute, Beijing 100854, People's Republic of China
| | - Tzyy-Ping Jung
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Swartz Centre for Computational Neuroscience, University of California, San Diego, CA, United States of America
| | - Dong Ming
- Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin 300072, People's Republic of China
- Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, People's Republic of China
| |
Collapse
|
5
|
Liu D, Xu X, Li D, Li J, Yu X, Ling Z, Hong B. Intracranial brain-computer interface spelling using localized visual motion response. Neuroimage 2022; 258:119363. [PMID: 35688315 DOI: 10.1016/j.neuroimage.2022.119363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 06/03/2022] [Accepted: 06/06/2022] [Indexed: 11/30/2022] Open
Abstract
Intracranial brain-computer interfaces (BCIs) can assist severely disabled persons in text communication and environmental control with high precision and speed. Nevertheless, sustainable BCI implants require minimal invasiveness. One of the implantation strategies is to adopt localized and robust cortical activities to drive BCI communication and to make a precise presurgical planning. The visual motion response is a good candidate for inclusion in this strategy because of its focal activity over the middle temporal visual area (MT). Here, we developed an intracranial BCI for spelling, utilizing only three electrodes over the MT area. The best recording electrodes were decided by preoperative functional magnetic resonance imaging (MRI) localization of the MT, and local neural activities were further enhanced by differential rereferencing of these electrodes. The BCI spelling system was validated both offline and online by five epilepsy patients, achieving the fastest speed of 62 bits/min, i.e., 12 characters/min. Moreover, the response patterns of dual-directional visual motion stimuli provided an additional dimension of BCI target encoding and paved the way for a higher information transfer rate of intracranial BCI spelling.
Collapse
Affiliation(s)
- Dingkun Liu
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, Beijing, 100084, China
| | - Xin Xu
- Department of Neurosurgery, Chinese PLA General Hospital, Beijing, Beijing, 100853, China
| | - Dongyang Li
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, Beijing, 100084, China
| | - Jie Li
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, Beijing, 100084, China
| | - Xinguang Yu
- Department of Neurosurgery, Chinese PLA General Hospital, Beijing, Beijing, 100853, China
| | - Zhipei Ling
- Department of Neurosurgery, Chinese PLA General Hospital, Beijing, Beijing, 100853, China
| | - Bo Hong
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, Beijing, 100084, China; McGovern Institute for Brain Research, Tsinghua University, Beijing, Beijing, 100084, China.
| |
Collapse
|
6
|
Zhou X, Xu M, Xiao X, Wang Y, Jung TP, Ming D. Detection of fixation points using a small visual landmark for brain-computer interfaces. J Neural Eng 2021; 18. [PMID: 34130268 DOI: 10.1088/1741-2552/ac0b51] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Accepted: 06/15/2021] [Indexed: 11/12/2022]
Abstract
Objective.The speed of visual brain-computer interfaces (v-BCIs) has been greatly improved in recent years. However, the traditional v-BCI paradigms require users to directly gaze at the intensive flickering items, which would cause severe problems such as visual fatigue and excessive visual resource consumption in practical applications. Therefore, it is imperative to develop a user-friendly v-BCI.Approach.According to the retina-cortical relationship, this study developed a novel BCI paradigm to detect the fixation point of eyes using a small visual stimulus that subtended only 0.6° in visual angle and was out of the central visual field. Specifically, the visual stimulus was treated as a landmark to judge the eccentricity and polar angle of the fixation point. Sixteen different fixation points were selected around the visual landmark, i.e. different combinations of two eccentricities (2° and 4°) and eight polar angles (0,π4,π2,3π4,π,5π4,3π2and7π4). Twelve subjects participated in this study, and they were asked to gaze at one out of the 16 points for each trial. A multi-class discriminative canonical pattern matching (Multi-DCPM) algorithm was proposed to decode the user's fixation point.Main results.We found the visual stimulation landmark elicited different spatial event-related potential patterns for different fixation points. Multi-DCPM could achieve an average accuracy of 66.2% with a standard deviation of 15.8% for the classification of the sixteen fixation points, which was significantly higher than traditional algorithms (p⩽0.001). Experimental results of this study demonstrate the feasibility of using a small visual stimulus as a landmark to track the relative position of the fixation point.Significance.The proposed new paradigm provides a potential approach to alleviate the problem of irritating stimuli in v-BCIs, which can broaden the applications of BCIs.
Collapse
Affiliation(s)
- Xiaoyu Zhou
- The Laboratory of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China
| | - Minpeng Xu
- The Laboratory of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China.,The Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
| | - Xiaolin Xiao
- The Laboratory of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China.,The Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
| | - Yijun Wang
- The State Key Laboratory on Integrated Optoelectronics, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, People's Republic of China
| | - Tzyy-Ping Jung
- The Laboratory of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China.,The Swartz Center for Computational Neuroscience, University of California, San Diego, CA, United States of America
| | - Dong Ming
- The Laboratory of Neural Engineering & Rehabilitation, Department of Biomedical Engineering, College of Precision Instruments and Optoelectronics Engineering, Tianjin University, Tianjin, People's Republic of China.,The Tianjin International Joint Research Center for Neural Engineering, Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, People's Republic of China
| |
Collapse
|
7
|
Liu D, Liu C, Chen J, Zhang D, Hong B. Doubling the Speed of N200 Speller via Dual-Directional Motion Encoding. IEEE Trans Biomed Eng 2020; 68:204-213. [PMID: 32746042 DOI: 10.1109/tbme.2020.3005518] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
OBJECTIVE Motion-onset visual evoked potentials (mVEPs)-based spellers, also known as N200 spellers, have been successfully implemented, avoiding flashing stimuli that are common in visual brain-computer interface (BCI). However, their information transfer rates (ITRs), typically below 50 bits/min, are lower than other visual BCI spellers. In this study, we sought to improve the speed of N200 speller to a level above the well-known P300 spellers. APPROACH Based on our finding of the spatio-temporal asymmetry of N200 response elicited by leftward and rightward visual motion, a novel dual-directional N200 speller was implemented. By presenting visual stimuli moving in two different directions simultaneously, the new paradigm reduced the stimuli presentation time by half, while ensuring separable N200 features between two visual motion directions. Furthermore, a probability-based dynamic stopping algorithm was also proposed to shorten the decision time for each output further. Both offline and online tests were conducted to evaluate the performance in ten participants. MAIN RESULTS Offline results revealed contralateral dominant temporal and spatial patterns in N200 responses when subjects attended to stimuli moving leftward or rightward. In online experiments, the dual-directional paradigm achieved an average ITR of 79.8 bits/min, with the highest ITR of 124.8 bits/min. Compared with the traditional uni-directional N200 speller, the median gain on the ITR was 202%. SIGNIFICANCE The proposed dual-directional paradigm managed to double the speed of the N200 speller. Together with its non-flashing characteristics, this dual-directional N200 speller is promising to be a competent candidate for fast and reliable BCI applications.
Collapse
|
8
|
Li Z, Liu K, Deng X, Wang G. Spatial fusion of maximum signal fraction analysis for frequency recognition in SSVEP-based BCI. Biomed Signal Process Control 2020. [DOI: 10.1016/j.bspc.2020.102042] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
9
|
Chen J, Hong B, Wang Y, Gao X, Zhang D. Towards a fully spatially coded brain-computer interface: simultaneous decoding of visual eccentricity and direction. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:3091-3094. [PMID: 31946541 DOI: 10.1109/embc.2019.8856586] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
By encoding visual targets with different locations relative to a stimulus, spatially coded brain-computer interface (BCI) has regained interest nowadays. Recent spatially coded BCI studies have demonstrated the feasibility of single-stimulus, multi-target BCIs, suggesting their potentials for simple and efficient applications. However, these studies have only decoded the visual direction information from the neural responses. To fully utilize the visual spatial information, it is necessary to include the visual eccentricity information as well. In the present study, the decodability of visual eccentricity information for BCI application was investigated for the first time. Sixteen targets were encoded simultaneously with eight directions and two eccentricities relative to a visual motion stimulus. Distinct neural spatial patterns and response strengths of motion-onset visual evoked potentials were elicited in the 16 attention conditions. The offline analysis reached an average classification accuracy of 63.1±11.5%, and the best-performing participant achieved an accuracy of 81.9%, well above the chance level (i.e., 6.25%) for 16-target classification. The results suggested the feasibility of simultaneous decoding of visual eccentricity and direction information towards a fully spatially coded BCI.
Collapse
|
10
|
Simultaneous Decoding of Eccentricity and Direction Information for a Single-Flicker SSVEP BCI. ELECTRONICS 2019. [DOI: 10.3390/electronics8121554] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The feasibility of a steady-state visual evoked potential (SSVEP) brain–computer interface (BCI) with a single-flicker stimulus for multiple-target decoding has been demonstrated in a number of recent studies. The single-flicker BCIs have mainly employed the direction information for encoding the targets, i.e., different targets are placed at different spatial directions relative to the flicker stimulus. The present study explored whether visual eccentricity information can also be used to encode targets for the purpose of increasing the number of targets in the single-flicker BCIs. A total number of 16 targets were encoded, placed at eight spatial directions, and two eccentricities (2.5° and 5°) relative to a 12 Hz flicker stimulus. Whereas distinct SSVEP topographies were elicited when participants gazed at targets of different directions, targets of different eccentricities were mainly represented by different signal-to-noise ratios (SNRs). Using a canonical correlation analysis-based classification algorithm, simultaneous decoding of both direction and eccentricity information was achieved, with an offline 16-class accuracy of 66.8 ± 16.4% averaged over 12 participants and a best individual accuracy of 90.0%. Our results demonstrate a single-flicker BCI with a substantially increased target number towards practical applications.
Collapse
|