1
|
He B, Zhang H, Qin T, Shi B, Wang Q, Dong W. A simultaneous EEG and eye-tracking dataset for remote sensing object detection. Sci Data 2025; 12:651. [PMID: 40246854 DOI: 10.1038/s41597-025-04995-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2024] [Accepted: 04/09/2025] [Indexed: 04/19/2025] Open
Abstract
We introduce the EEGET-RSOD, a simultaneous electroencephalography (EEG) and eye-tracking dataset for remote sensing object detection. This dataset contains EEG and eye-tracking data when 38 remote sensing experts located specific objects in 1,000 remote sensing images within a limited time frame. This task reflects the typical cognitive processes associated with human visual search and object identification in remote sensing imagery. To our knowledge, EEGET-RSOD is the first publicly available dataset to offer synchronized eye-tracking and EEG data for remote sensing images. This dataset will not only advance the study of human visual cognition in real-world environment, but also bridge the gap between human cognition and artificial intelligence, enhancing the interpretability and reliability of AI models in geospatial applications.
Collapse
Affiliation(s)
- Bing He
- Advanced Interdisciplinary Institute of Satellite Applications, State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, China
| | - Hongqiang Zhang
- Advanced Interdisciplinary Institute of Satellite Applications, State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, China
| | - Tong Qin
- Research Group CartoGIS, Department of Geography, Ghent University, Ghent, Belgium
| | - Bowen Shi
- Department of Land Surveying and Geo-Informatics, The Hong Kong Polytechnic University, Hong Kong, China
| | - Qiao Wang
- Advanced Interdisciplinary Institute of Satellite Applications, State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, China
| | - Weihua Dong
- Advanced Interdisciplinary Institute of Satellite Applications, State Key Laboratory of Earth Surface Processes and Resource Ecology, Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, China.
| |
Collapse
|
2
|
Li X, Wei W, Qiu S, He H. A temporal-spectral fusion transformer with subject-specific adapter for enhancing RSVP-BCI decoding. Neural Netw 2025; 181:106844. [PMID: 39509814 DOI: 10.1016/j.neunet.2024.106844] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2024] [Revised: 10/01/2024] [Accepted: 10/23/2024] [Indexed: 11/15/2024]
Abstract
The Rapid Serial Visual Presentation (RSVP)-based Brain-Computer Interface (BCI) is an efficient technology for target retrieval using electroencephalography (EEG) signals. The performance improvement of traditional decoding methods relies on a substantial amount of training data from new test subjects, which increases preparation time for BCI systems. Several studies introduce data from existing subjects to reduce the dependence of performance improvement on data from new subjects, but their optimization strategy based on adversarial learning with extensive data increases training time during the preparation procedure. Moreover, most previous methods only focus on the single-view information of EEG signals, but ignore the information from other views which may further improve performance. To enhance decoding performance while reducing preparation time, we propose a Temporal-Spectral fusion transformer with Subject-specific Adapter (TSformer-SA). Specifically, a cross-view interaction module is proposed to facilitate information transfer and extract common representations across two-view features extracted from EEG temporal signals and spectrogram images. Then, an attention-based fusion module fuses the features of two views to obtain comprehensive discriminative features for classification. Furthermore, a multi-view consistency loss is proposed to maximize the feature similarity between two views of the same EEG signal. Finally, we propose a subject-specific adapter to rapidly transfer the knowledge of the model trained on data from existing subjects to decode data from new subjects. Experimental results show that TSformer-SA significantly outperforms comparison methods and achieves outstanding performance with limited training data from new subjects. This facilitates efficient decoding and rapid deployment of BCI systems in practical use.
Collapse
Affiliation(s)
- Xujin Li
- Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China; School of Future Technology, University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China
| | - Wei Wei
- Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Shuang Qiu
- Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China; School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China.
| | - Huiguang He
- Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China; School of Future Technology, University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China; School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China.
| |
Collapse
|
3
|
Yuan Z, Zhou Q, Wang B, Zhang Q, Yang Y, Zhao Y, Guo Y, Zhou J, Wang C. PSAEEGNet: pyramid squeeze attention mechanism-based CNN for single-trial EEG classification in RSVP task. Front Hum Neurosci 2024; 18:1385360. [PMID: 38756843 PMCID: PMC11097777 DOI: 10.3389/fnhum.2024.1385360] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2024] [Accepted: 04/08/2024] [Indexed: 05/18/2024] Open
Abstract
Introduction Accurate classification of single-trial electroencephalogram (EEG) is crucial for EEG-based target image recognition in rapid serial visual presentation (RSVP) tasks. P300 is an important component of a single-trial EEG for RSVP tasks. However, single-trial EEG are usually characterized by low signal-to-noise ratio and limited sample sizes. Methods Given these challenges, it is necessary to optimize existing convolutional neural networks (CNNs) to improve the performance of P300 classification. The proposed CNN model called PSAEEGNet, integrates standard convolutional layers, pyramid squeeze attention (PSA) modules, and deep convolutional layers. This approach arises the extraction of temporal and spatial features of the P300 to a finer granularity level. Results Compared with several existing single-trial EEG classification methods for RSVP tasks, the proposed model shows significantly improved performance. The mean true positive rate for PSAEEGNet is 0.7949, and the mean area under the receiver operating characteristic curve (AUC) is 0.9341 (p < 0.05). Discussion These results suggest that the proposed model effectively extracts features from both temporal and spatial dimensions of P300, leading to a more accurate classification of single-trial EEG during RSVP tasks. Therefore, this model has the potential to significantly enhance the performance of target recognition systems based on EEG, contributing to the advancement and practical implementation of target recognition in this field.
Collapse
Affiliation(s)
- Zijian Yuan
- School of Intelligent Medicine and Biotechnology, Guilin Medical University, Guangxi, China
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Qian Zhou
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Baozeng Wang
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Qi Zhang
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Yang Yang
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Yuwei Zhao
- Beijing Institute of Basic Medical Sciences, Beijing, China
| | - Yong Guo
- School of Intelligent Medicine and Biotechnology, Guilin Medical University, Guangxi, China
| | - Jin Zhou
- Beijing Institute of Basic Medical Sciences, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Changyong Wang
- Beijing Institute of Basic Medical Sciences, Beijing, China
| |
Collapse
|
4
|
Chen Z, Yang R, Huang M, Li F, Lu G, Wang Z. EEGProgress: A fast and lightweight progressive convolution architecture for EEG classification. Comput Biol Med 2024; 169:107901. [PMID: 38159400 DOI: 10.1016/j.compbiomed.2023.107901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 12/11/2023] [Accepted: 12/23/2023] [Indexed: 01/03/2024]
Abstract
Because of the intricate topological structure and connection of the human brain, extracting deep spatial features from electroencephalograph (EEG) signals is a challenging and time-consuming task. The extraction of topological spatial information plays a crucial role in EEG classification, and the architecture of the spatial convolution greatly affects the performance and complexity of convolutional neural network (CNN) based EEG classification models. In this study, a progressive convolution CNN architecture named EEGProgress is proposed, aiming to efficiently extract the topological spatial information of EEG signals from multi-scale levels (electrode, brain region, hemisphere, global) with superior speed. To achieve this, the raw EEG data is permuted using the empirical topological permutation rule, integrating the EEG data with numerous topological properties. Subsequently, the spatial features are extracted by a progressive feature extractor including prior, electrode, region, and hemisphere convolution blocks, progressively extracting the deep spatial features with reduced parameters and speed. Finally, the comparison and ablation experiments under both cross-subject and within-subject scenarios are conducted on a public dataset to verify the performance of the proposed EEGProgress and the effectiveness of the topological permutation. The results demonstrate the superior feature extraction ability of the proposed EEGProgress, with an average increase of 4.02% compared to other CNN-based EEG classification models under both cross-subject and within-subject scenarios. Furthermore, with the obtained average testing time, FLOPs, and parameters, the proposed EEGProgress outperforms other comparison models in terms of model complexity.
Collapse
Affiliation(s)
- Zhige Chen
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China; School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool L69 3BX, United Kingdom
| | - Rui Yang
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China
| | - Mengjie Huang
- Design School, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China.
| | - Fumin Li
- School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215123, China; School of Electrical Engineering, Electronics and Computer Science, University of Liverpool, Liverpool L69 3BX, United Kingdom
| | - Guoping Lu
- School of Electrical Engineering, Nantong University, Nantong 226019, China
| | - Zidong Wang
- Department of Computer Science, Brunel University London, Uxbridge, Middlesex UB8 3PH, United Kingdom
| |
Collapse
|
5
|
Lian J, Qiao X, Zhao Y, Li S, Wang C, Zhou J. EEG-Based Target Detection Using an RSVP Paradigm under Five Levels of Weak Hidden Conditions. Brain Sci 2023; 13:1583. [PMID: 38002543 PMCID: PMC10670035 DOI: 10.3390/brainsci13111583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 11/06/2023] [Accepted: 11/07/2023] [Indexed: 11/26/2023] Open
Abstract
Although target detection based on electroencephalogram (EEG) signals has been extensively investigated recently, EEG-based target detection under weak hidden conditions remains a problem. In this paper, we proposed a rapid serial visual presentation (RSVP) paradigm for target detection corresponding to five levels of weak hidden conditions quantitively based on the RGB color space. Eighteen subjects participated in the experiment, and the neural signatures, including P300 amplitude and latency, were investigated. Detection performance was evaluated under five levels of weak hidden conditions using the linear discrimination analysis and support vector machine classifiers on different channel sets. The experimental results showed that, compared with the benchmark condition, (1) the P300 amplitude significantly decreased (8.92 ± 1.24 μV versus 7.84 ± 1.40 μV, p = 0.021) and latency was significantly prolonged (582.39 ± 25.02 ms versus 643.83 ± 26.16 ms, p = 0.028) only under the weakest hidden condition, and (2) the detection accuracy decreased by less than 2% (75.04 ± 3.24% versus 73.35 ± 3.15%, p = 0.029) with a more than 90% reduction in channel number (62 channels versus 6 channels), determined using the proposed channel selection method under the weakest hidden condition. Our study can provide new insights into target detection under weak hidden conditions based on EEG signals with a rapid serial visual presentation paradigm. In addition, it may expand the application of brain-computer interfaces in EEG-based target detection areas.
Collapse
Affiliation(s)
- Jinling Lian
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
| | - Xin Qiao
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
| | - Yuwei Zhao
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
| | - Siwei Li
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
| | - Changyong Wang
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
| | - Jin Zhou
- Department of Neural Engineering and Biological Interdisciplinary Studies, Beijing Institute of Basic Medical Sciences, 27 Taiping Rd., Beijing 100850, China; (J.L.); (X.Q.); (Y.Z.); (S.L.)
- Chinese Institute for Brain Research, Zhongguancun Life Science Park, Changping District, Beijing 102206, China
| |
Collapse
|