STAnet: A Spatiotemporal Attention Network for Decoding Auditory Spatial Attention from EEG

Vegetation change detection and recovery assessment based on post-fire satellite imagery using deep learning

Wildfires are uncontrolled fires fuelled by dry conditions, high winds, and flammable materials that profoundly impact vegetation, leading to significant consequences including noteworthy changes to ecosystems. In this study, we provide a novel methodology to understand and evaluate post-fire effects on vegetation. In regions affected by wildfires, earth-observation data from various satellite sources can be vital in monitoring vegetation and assessing its impact. These effects can be understood by detecting vegetation change over the years using a novel unsupervised method termed Deep Embedded Clustering (DEC), which enables us to classify regions based on whether there has been a change in vegetation after the fire. Our model achieves an impressive accuracy of 96.17%. Appropriate vegetation indices can be used to evaluate the evolution of vegetation patterns over the years; for this study, we utilized Enhanced Vegetation Index (EVI) based trend analysis showing the greening fraction, which ranges from 0.1 to 22.4 km2 while the browning fraction ranges from 0.1 to 18.1 km2 over the years. Vegetation recovery maps can be created to assess re-vegetation in regions affected by the fire, which is performed via a deep learning-based unsupervised method, Adaptive Generative Adversarial Neural Network Model (AdaptiGAN) on post-fire data collected from various regions affected by wildfire with a training error of 0.075 proving its capability. Based on the results obtained from the study, our approach tends to have notable merits when compared to pre-existing works.

Attention-guided graph structure learning network for EEG-enabled auditory attention detection

Objective: Decoding auditory attention from brain signals is essential for the development of neuro-steered hearing aids. This study aims to overcome the challenges of extracting discriminative feature representations from electroencephalography (EEG) signals for auditory attention detection (AAD) tasks, particularly focusing on the intrinsic relationships between different EEG channels.Approach: We propose a novel attention-guided graph structure learning network, AGSLnet, which leverages potential relationships between EEG channels to improve AAD performance. Specifically, AGSLnet is designed to dynamically capture latent relationships between channels and construct a graph structure of EEG signals.Main result: We evaluated AGSLnet on two publicly available AAD datasets and demonstrated its superiority and robustness over state-of-the-art models. Visualization of the graph structure trained by AGSLnet supports previous neuroscience findings, enhancing our understanding of the underlying neural mechanisms.Significance: This study presents a novel approach for examining brain functional connections, improving AAD performance in low-latency settings, and supporting the development of neuro-steered hearing aids.

Deep learning-based auditory attention decoding in listeners with hearing impairment. J Neural Eng 2024;21:036022. [PMID: 38729132 DOI: 10.1088/1741-2552/ad49d7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Objective.This study develops a deep learning (DL) method for fast auditory attention decoding (AAD) using electroencephalography (EEG) from listeners with hearing impairment (HI). It addresses three classification tasks: differentiating noise from speech-in-noise, classifying the direction of attended speech (left vs. right) and identifying the activation status of hearing aid noise reduction algorithms (OFF vs. ON). These tasks contribute to our understanding of how hearing technology influences auditory processing in the hearing-impaired population.Approach.Deep convolutional neural network (DCNN) models were designed for each task. Two training strategies were employed to clarify the impact of data splitting on AAD tasks: inter-trial, where the testing set used classification windows from trials that the training set had not seen, and intra-trial, where the testing set used unseen classification windows from trials where other segments were seen during training. The models were evaluated on EEG data from 31 participants with HI, listening to competing talkers amidst background noise.Main results.Using 1 s classification windows, DCNN models achieve accuracy (ACC) of 69.8%, 73.3% and 82.9% and area-under-curve (AUC) of 77.2%, 80.6% and 92.1% for the three tasks respectively on inter-trial strategy. In the intra-trial strategy, they achieved ACC of 87.9%, 80.1% and 97.5%, along with AUC of 94.6%, 89.1%, and 99.8%. Our DCNN models show good performance on short 1 s EEG samples, making them suitable for real-world applications. Conclusion: Our DCNN models successfully addressed three tasks with short 1 s EEG windows from participants with HI, showcasing their potential. While the inter-trial strategy demonstrated promise for assessing AAD, the intra-trial approach yielded inflated results, underscoring the important role of proper data splitting in EEG-based AAD tasks.Significance.Our findings showcase the promising potential of EEG-based tools for assessing auditory attention in clinical contexts and advancing hearing technology, while also promoting further exploration of alternative DL architectures and their potential constraints.

DAST: A Domain-Adaptive Learning Combining Spatio-Temporal Dynamic Attention for Electroencephalography Emotion Recognition

Multimodal emotion recognition with EEG-based have become mainstream in affective computing. However, previous studies mainly focus on perceived emotions (including posture, speech or face expression et al.) of different subjects, while the lack of research on induced emotions (including video or music et al.) limited the development of two-ways emotions. To solve this problem, we propose a multimodal domain adaptive method based on EEG and music called the DAST, which uses spatio-temporal adaptive attention (STA-attention) to globally model the EEG and maps all embeddings dynamically into high-dimensionally space by adaptive space encoder (ASE). Then, adversarial training is performed with domain discriminator and ASE to learn invariant emotion representations. Furthermore, we conduct extensive experiments on the DEAP dataset, and the results show that our method can further explore the relationship between induced and perceived emotions, and provide a reliable reference for exploring the potential correlation between EEG and music stimulation.

TBEEG: A Two-Branch Manifold Domain Enhanced Transformer Algorithm for Learning EEG Decoding

The electroencephalogram-based (EEG) brain-computer interface (BCI) has garnered significant attention in recent research. However, the practicality of EEG remains constrained by the lack of efficient EEG decoding technology. The challenge lies in effectively translating intricate EEG into meaningful, generalizable information. EEG signal decoding primarily relies on either time domain or frequency domain information. There lacks a method capable of simultaneously and effectively extracting both time and frequency domain features, as well as efficiently fuse these features. Addressing these limitations, a two-branch Manifold Domain enhanced transformer algorithm is designed to holistically capture EEG's spatio-temporal information. Our method projects the time-domain information of EEG signals into the Riemannian spaces to fully decode the time dependence of EEG signals. Using wavelet transform, the time domain information is converted into frequency domain information, and the spatial information contained in the frequency domain information of EEG signal is mined through the spectrogram. The effectiveness of the proposed TBEEG algorithm is validated on BCIC-IV-2a dataset and MAMEM-SSVEP-II datasets.

Research on bronze wine vessel classification using improved SSA-CBAM-GNNs

This article proposes an advanced classification algorithm for bronze drinking utensils, taking into account the complexity of their cultural characteristics and the challenges of dynasty classification. The SSA-CBAM-GNNs algorithm integrates the Sparrow Search Algorithm (SSA), Spatial and Spectral Attention (CBAM) modules, and Graph Neural Networks (GNNs). The CBAM module is essential for optimizing feature extraction weights in graph neural networks, while SSA enhances the weighted network and expedites the convergence process. Experimental results, validated through various performance evaluation indicators, illustrate the outstanding performance of the improved SSA-CBAM-GNNs algorithm in accurately identifying and classifying cultural features of bronze drinking utensils. Comparative experiments confirm the algorithm's superiority over other methods. Overall, this study proposes a highly efficient identification and classification algorithm, and its effectiveness and excellence in extracting and identifying cultural features of bronze drinking utensils are experimentally demonstrated.

What are wereallydecoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention

Objective.Spatial auditory attention decoding (Sp-AAD) refers to the task of identifying the direction of the speaker to which a person is attending in a multi-talker setting, based on the listener's neural recordings, e.g. electroencephalography (EEG). The goal of this study is to thoroughly investigate potential biases when training such Sp-AAD decoders on EEG data, particularly eye-gaze biases and latent trial-dependent confounds, which may result in Sp-AAD models that decode eye-gaze or trial-specific fingerprints rather than spatial auditory attention.Approach.We designed a two-speaker audiovisual Sp-AAD protocol in which the spatial auditory and visual attention were enforced to be either congruent or incongruent, and we recorded EEG data from sixteen participants undergoing several trials recorded at distinct timepoints. We trained a simple linear model for Sp-AAD based on common spatial patterns filters in combination with either linear discriminant analysis (LDA) or k-means clustering, and evaluated them both across- and within-trial.Main results.We found that even a simple linear Sp-AAD model is susceptible to overfitting to confounding signal patterns such as eye-gaze and trial fingerprints (e.g. due to feature shifts across trials), resulting in artificially high decoding accuracies. Furthermore, we found that changes in the EEG signal statistics across trials deteriorate the trial generalization of the classifier, even when the latter is retrained on the test trial with an unsupervised algorithm.Significance.Collectively, our findings confirm that there exist subtle biases and confounds that can strongly interfere with the decoding of spatial auditory attention from EEG. It is expected that more complicated non-linear models based on deep neural networks, which are often used for Sp-AAD, are even more vulnerable to such biases. Future work should perform experiments and model evaluations that avoid and/or control for such biases in Sp-AAD tasks.

Improving EEG-based decoding of the locus of auditory attention through domain adaptation. J Neural Eng 2023;20:066022. [PMID: 37988748 DOI: 10.1088/1741-2552/ad0e7b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Objective.This paper presents a novel domain adaptation (DA) framework to enhance the accuracy of electroencephalography (EEG)-based auditory attention classification, specifically for classifying the direction (left or right) of attended speech. The framework aims to improve the performances for subjects with initially low classification accuracy, overcoming challenges posed by instrumental and human factors. Limited dataset size, variations in EEG data quality due to factors such as noise, electrode misplacement or subjects, and the need for generalization across different trials, conditions and subjects necessitate the use of DA methods. By leveraging DA methods, the framework can learn from one EEG dataset and adapt to another, potentially resulting in more reliable and robust classification models.Approach.This paper focuses on investigating a DA method, based on parallel transport, for addressing the auditory attention classification problem. The EEG data utilized in this study originates from an experiment where subjects were instructed to selectively attend to one of the two spatially separated voices presented simultaneously.Main results.Significant improvement in classification accuracy was observed when poor data from one subject was transported to the domain of good data from different subjects, as compared to the baseline. The mean classification accuracy for subjects with poor data increased from 45.84% to 67.92%. Specifically, the highest achieved classification accuracy from one subject reached 83.33%, a substantial increase from the baseline accuracy of 43.33%.Significance.The findings of our study demonstrate the improved classification performances achieved through the implementation of DA methods. This brings us a step closer to leveraging EEG in neuro-steered hearing devices.

RGCnet: An Efficient Recursive Gated Convolutional Network for EEG-based Auditory Attention Detection

Humans are able to listen to one speaker and disregard others in a speaking crowd, referred to as the "cocktail party effect". EEG-based auditory attention detection (AAD) seeks to identify whom a listener is listening to by decoding one's EEG signals. Recent research has demonstrated that the self-attention mechanism is effective for AAD. In this paper, we present the Recursive Gated Convolutional network (RGCnet) for AAD, which implements long-range and high-order interactions as a self-attention mechanism, while maintaining a low computational cost. The RGCnet expands the 2nd order feature interactions to a higher order to model the complex interactions between EEG features. We evaluate RGCnet on two public datasets and compare it with other AAD models. Our results demonstrate that RGCnet outperforms other comparative models under various conditions, thus potentially improving the control of neuro-steered hearing devices.