A Bio-Inspired Spiking Attentional Neural Network for Attentional Selection in the Listening Brain

Humans show a remarkable ability in solving the cocktail party problem. Decoding auditory attention from the brain signals is a major step toward the development of bionic ears emulating human capabilities. Electroencephalography (EEG)-based auditory attention detection (AAD) has attracted considerable interest recently. Despite much progress, the performance of traditional AAD decoders remains to be improved, especially in low-latency settings. State-of-the-art AAD decoders based on deep neural networks generally lack the intrinsic temporal coding ability in biological networks. In this study, we first propose a bio-inspired spiking attentional neural network, denoted as BSAnet, for decoding auditory attention. BSAnet is capable of exploiting the temporal dynamics of EEG signals using biologically plausible neurons and an attentional mechanism. Experiments on two publicly available datasets confirm the superior performance of BSAnet over other state-of-the-art systems across various evaluation conditions. Moreover, BSAnet imitates realistic brain-like information processing, through which we show the advantage of brain-inspired computational models.

Brain Topology Modeling With EEG-Graphs for Auditory Spatial Attention Detection

OBJECTIVE

Despite recent advances, the decoding of auditory attention from brain signals remains a challenge. A key solution is the extraction of discriminative features from high-dimensional data, such as multi-channel electroencephalography (EEG). However, to our knowledge, topological relationships between individual channels have not yet been considered in any study. In this work, we introduced a novel architecture that exploits the topology of the human brain to perform auditory spatial attention detection (ASAD) from EEG signals.

METHODS

We propose EEG-Graph Net, an EEG-graph convolutional network, which employs a neural attention mechanism. This mechanism models the topology of the human brain in terms of the spatial pattern of EEG signals as a graph. In the EEG-Graph, each EEG channel is represented by a node, while the relationship between two EEG channels is represented by an edge between the respective nodes. The convolutional network takes the multi-channel EEG signals as a time series of EEG-graphs and learns the node and edge weights from the contribution of the EEG signals to the ASAD task. The proposed architecture supports the interpretation of the experimental results by data visualization.

RESULTS

We conducted experiments on two publicly available databases. The experimental results showed that EEG-Graph Net significantly outperforms the state-of-the-art methods in terms of decoding performance. In addition, the analysis of the learned weight patterns provides insights into the processing of continuous speech in the brain and confirms findings from neuroscientific studies.

CONCLUSION

We showed that modeling brain topology with EEG-graphs yields highly competitive results for auditory spatial attention detection.

SIGNIFICANCE

The proposed EEG-Graph Net is more lightweight and accurate than competing baselines and provides explanations for the results. Also, the architecture can be easily transferred to other brain-computer interface (BCI) tasks.

Doppler Frequency-Shift Information Processing in WOx -Based Memristive Synapse for Auditory Motion Perception

Auditory motion perception is one crucial capability to decode and discriminate the spatiotemporal information for neuromorphic auditory systems. Doppler frequency-shift feature and interaural time difference (ITD) are two fundamental cues of auditory information processing. In this work, the functions of azimuth detection and velocity detection, as the typical auditory motion perception, are demonstrated in a WO_x -based memristive synapse. The WO_x memristor presents both the volatile mode (M1) and semi-nonvolatile mode (M2), which are capable of implementing the high-pass filtering and processing the spike trains with a relative timing and frequency shift. In particular, the Doppler frequency-shift information processing for velocity detection is emulated in the WO_x memristor based auditory system for the first time, which relies on a scheme of triplet spike-timing-dependent-plasticity in the memristor. These results provide new opportunities for the mimicry of auditory motion perception and enable the auditory sensory system to be applied in future neuromorphic sensing.

STAnet: A Spatiotemporal Attention Network for Decoding Auditory Spatial Attention from EEG

OBJECTIVE

Humans are able to localize the source of a sound. This enables them to direct attention to a particular speaker in a cocktail party. Psycho-acoustic studies show that the sensory cortices of the human brain respond to the location of sound sources differently, and the auditory attention itself is a dynamic and temporally based brain activity. In this work, we seek to build a computational model which uses both spatial and temporal information manifested in EEG signals for auditory spatial attention detection (ASAD).

METHODS

We propose an end-to-end spatiotemporal attention network, denoted as STAnet, to detect auditory spatial attention from EEG. The STAnet is designed to assign differentiated weights dynamically to EEG channels through a spatial attention mechanism, and to temporal patterns in EEG signals through a temporal attention mechanism.

RESULTS

We report the ASAD experiments on two publicly available datasets. The STAnet outperforms other competitive models by a large margin under various experimental conditions. Its attention decision for 1-second decision window outperforms that of the state-of-the-art techniques for 10-second decision window. Experimental results also demonstrate that the STAnet achieves competitive performance on EEG signals ranging from 64 to as few as 16 channels.

CONCLUSION

This study provides evidence suggesting that efficient low-density EEG online decoding is within reach.

SIGNIFICANCE

This study also marks an important step towards the practical implementation of ASAD in real life applications.

Ear-Specific Hemispheric Asymmetry in Unilateral Deafness Revealed by Auditory Cortical Activity

Profound unilateral deafness reduces the ability to localize sounds achieved via binaural hearing. Furthermore, unilateral deafness promotes a substantial change in cortical processing to binaural stimulation, thereby leading to reorganization over the whole brain. Although distinct patterns in the hemispheric laterality depending on the side and duration of deafness have been suggested, the neurological mechanisms underlying the difference in relation to behavioral performance when detecting spatially varied cues remain unknown. To elucidate the mechanism, we compared N1/P2 auditory cortical activities and the pattern of hemispheric asymmetry of normal hearing, unilaterally deaf (UD), and simulated acute unilateral hearing loss groups while passively listening to speech sounds delivered from different locations under open free field condition. The behavioral performances of the participants concerning sound localization were measured by detecting sound sources in the azimuth plane. The results reveal a delayed reaction time in the right-sided UD (RUD) group for the sound localization task and prolonged P2 latency compared to the left-sided UD (LUD) group. Moreover, the RUD group showed adaptive cortical reorganization evidenced by increased responses in the hemisphere ipsilateral to the intact ear for individuals with better sound localization whereas left-sided unilateral deafness caused contralateral dominance in activity from the hearing ear. The brain dynamics of right-sided unilateral deafness indicate greater capability of adaptive change to compensate for impairment in spatial hearing. In addition, cortical N1 responses to spatially varied speech sounds in unilateral deaf people were inversely related to the duration of deafness in the area encompassing the right auditory cortex, indicating that early intervention would be needed to protect from maladaptation of the central auditory system following unilateral deafness.

Shared and modality-specific brain regions that mediate auditory and visual word comprehension

Visual speech carried by lip movements is an integral part of communication. Yet, it remains unclear in how far visual and acoustic speech comprehension are mediated by the same brain regions. Using multivariate classification of full-brain MEG data, we first probed where the brain represents acoustically and visually conveyed word identities. We then tested where these sensory-driven representations are predictive of participants' trial-wise comprehension. The comprehension-relevant representations of auditory and visual speech converged only in anterior angular and inferior frontal regions and were spatially dissociated from those representations that best reflected the sensory-driven word identity. These results provide a neural explanation for the behavioural dissociation of acoustic and visual speech comprehension and suggest that cerebral representations encoding word identities may be more modality-specific than often upheld.

Where is the cocktail party? Decoding locations of attended and unattended moving sound sources using EEG

Recently, we showed that in a simple acoustic scene with one sound source, auditory cortex tracks the time-varying location of a continuously moving sound. Specifically, we found that both the delta phase and alpha power of the electroencephalogram (EEG) can be used to reconstruct the sound source azimuth. However, in natural settings, we are often presented with a mixture of multiple competing sounds and so we must focus our attention on the relevant source in order to segregate it from the competing sources e.g. 'cocktail party effect'. While many studies have examined this phenomenon in the context of sound envelope tracking by the cortex, it is unclear how we process and utilize spatial information in complex acoustic scenes with multiple sound sources. To test this, we created an experiment where subjects listened to two concurrent sound stimuli that were moving within the horizontal plane over headphones while we recorded their EEG. Participants were tasked with paying attention to one of the two presented stimuli. The data were analyzed by deriving linear mappings, temporal response functions (TRF), between EEG data and attended as well unattended sound source trajectories. Next, we used these TRFs to reconstruct both trajectories from previously unseen EEG data. In a first experiment we used noise stimuli and included the task involved spatially localizing embedded targets. Then, in a second experiment, we employed speech stimuli and a non-spatial speech comprehension task. Results showed the trajectory of an attended sound source can be reliably reconstructed from both delta phase and alpha power of EEG even in the presence of distracting stimuli. Moreover, the reconstruction was robust to task and stimulus type. The cortical representation of the unattended source position was below detection level for the noise stimuli, but we observed weak tracking of the unattended source location for the speech stimuli by the delta phase of EEG. In addition, we demonstrated that the trajectory reconstruction method can in principle be used to decode selective attention on a single-trial basis, however, its performance was inferior to envelope-based decoders. These results suggest a possible dissociation of delta phase and alpha power of EEG in the context of sound trajectory tracking. Moreover, the demonstrated ability to localize and determine the attended speaker in complex acoustic environments is particularly relevant for cognitively controlled hearing devices.

Neural tracking of auditory motion is reflected by delta phase and alpha power of EEG

It is of increasing practical interest to be able to decode the spatial characteristics of an auditory scene from electrophysiological signals. However, the cortical representation of auditory space is not well characterized, and it is unclear how cortical activity reflects the time-varying location of a moving sound. Recently, we demonstrated that cortical response measures to discrete noise bursts can be decoded to determine their origin in space. Here we build on these findings to investigate the cortical representation of a continuously moving auditory stimulus using scalp recorded electroencephalography (EEG). In a first experiment, subjects listened to pink noise over headphones which was spectro-temporally modified to be perceived as randomly moving on a semi-circular trajectory in the horizontal plane. While subjects listened to the stimuli, we recorded their EEG using a 128-channel acquisition system. The data were analysed by 1) building a linear regression model (decoder) mapping the relationship between the stimulus location and a training set of EEG data, and 2) using the decoder to reconstruct an estimate of the time-varying sound source azimuth from the EEG data. The results showed that we can decode sound trajectory with a reconstruction accuracy significantly above chance level. Specifically, we found that the phase of delta (<2 Hz) and power of alpha (8-12 Hz) EEG track the dynamics of a moving auditory object. In a follow-up experiment, we replaced the noise with pulse train stimuli containing only interaural level and time differences (ILDs and ITDs respectively). This allowed us to investigate whether our trajectory decoding is sensitive to both acoustic cues. We found that the sound trajectory can be decoded for both ILD and ITD stimuli. Moreover, their neural signatures were similar and even allowed successful cross-cue classification. This supports the notion of integrated processing of ILD and ITD at the cortical level. These results are particularly relevant for application in devices such as cognitively controlled hearing aids and for the evaluation of virtual acoustic environments.