Selective auditory attention detection based on effective connectivity by single-trial EEG

A GRU-CNN model for auditory attention detection using microstate and recurrence quantification analysis

Attention as a cognition ability plays a crucial role in perception which helps humans to concentrate on specific objects of the environment while discarding others. In this paper, auditory attention detection (AAD) is investigated using different dynamic features extracted from multichannel electroencephalography (EEG) signals when listeners attend to a target speaker in the presence of a competing talker. To this aim, microstate and recurrence quantification analysis are utilized to extract different types of features that reflect changes in the brain state during cognitive tasks. Then, an optimized feature set is determined by employing the processes of significant feature selection based on classification performance. The classifier model is developed by hybrid sequential learning that employs Gated Recurrent Units (GRU) and Convolutional Neural Network (CNN) into a unified framework for accurate attention detection. The proposed AAD method shows that the selected feature set achieves the most discriminative features for the classification process. Also, it yields the best performance as compared with state-of-the-art AAD approaches from the literature in terms of various measures. The current study is the first to validate the use of microstate and recurrence quantification parameters to differentiate auditory attention using reinforcement learning without access to stimuli.

Physical Exercise Effects on University Students’ Attention: An EEG Analysis Approach

Physically active breaks (AB) are currently being proposed as an interesting tool to improve students’ attention. Reviews and meta-analyses confirm their effect on attention, but also warned about the sparse evidence based on vigilance and university students. Therefore, this pilot study aimed to (a) determine the effects of AB in comparison with passive breaks on university students’ vigilance and (b) to validate an analysis model based on machine learning algorithms in conjunction with a multiparametric model based on electroencephalography (EEG) signal features. Through a counterbalanced within-subject experimental study, six university students (two female; mean age = 25.67, STD = 3.61) had their vigilance performances (i.e., response time in Psycho-Motor Vigilance Task) and EEG measured, before and after a lecture with an AB and another lecture with a passive break. A multiparametric model based on the spectral power, signal entropy and response time has been developed. Furthermore, this model, together with different machine learning algorithms, shows that for the taken signals there are significant differences after the AB lesson, implying an improvement in attention. These differences are most noticeable with the SVM with RBF kernel and ANNs with F1-score of 85% and 88%, respectively. In conclusion, results showed that students performed better on vigilance after the lecture with AB. Although limited, the evidence found could help researchers to be more accurate in their EEG analyses and lecturers and teachers to improve their students’ attentions in a proper way.

A Speech-Level–Based Segmented Model to Decode the Dynamic Auditory Attention States in the Competing Speaker Scenes

In the competing speaker environments, human listeners need to focus or switch their auditory attention according to dynamic intentions. The reliable cortical tracking ability to the speech envelope is an effective feature for decoding the target speech from the neural signals. Moreover, previous studies revealed that the root mean square (RMS)–level–based speech segmentation made a great contribution to the target speech perception with the modulation of sustained auditory attention. This study further investigated the effect of the RMS-level–based speech segmentation on the auditory attention decoding (AAD) performance with both sustained and switched attention in the competing speaker auditory scenes. Objective biomarkers derived from the cortical activities were also developed to index the dynamic auditory attention states. In the current study, subjects were asked to concentrate or switch their attention between two competing speaker streams. The neural responses to the higher- and lower-RMS-level speech segments were analyzed via the linear temporal response function (TRF) before and after the attention switching from one to the other speaker stream. Furthermore, the AAD performance decoded by the unified TRF decoding model was compared to that by the speech-RMS-level–based segmented decoding model with the dynamic change of the auditory attention states. The results showed that the weight of the typical TRF component approximately 100-ms time lag was sensitive to the switching of the auditory attention. Compared to the unified AAD model, the segmented AAD model improved attention decoding performance under both the sustained and switched auditory attention modulations in a wide range of signal-to-masker ratios (SMRs). In the competing speaker scenes, the TRF weight and AAD accuracy could be used as effective indicators to detect the changes of the auditory attention. In addition, with a wide range of SMRs (i.e., from 6 to –6 dB in this study), the segmented AAD model showed the robust decoding performance even with short decision window length, suggesting that this speech-RMS-level–based model has the potential to decode dynamic attention states in the realistic auditory scenarios.

Ear-EEG-based binaural speech enhancement (ee-BSE) using auditory attention detection and audiometric characteristics of hearing-impaired subjects

Objective. Speech perception in cocktail party scenarios has been the concern of a group of researchers who are involved with the design of hearing-aid devices.Approach. In this paper, a new unified ear-EEG-based binaural speech enhancement system is introduced for hearing-impaired (HI) listeners. The proposed model, which is based on auditory attention detection (AAD) and individual hearing threshold (HT) characteristics, has four main processing stages. In the binaural processing stage, a system based on the deep neural network is trained to estimate auditory ratio masks for each of the speakers in the mixture signal. In the EEG processing stage, AAD is employed to select one ratio mask corresponding to the attended speech. Here, the same EEG data is also used to predict the HTs of listeners who participated in the EEG recordings. The third stage, called insertion gain computation, concerns the calculation of a special amplification gain based on individual HTs. Finally, in the selection-resynthesis-amplification stage, the attended speech signals of the target are resynthesized based on the selected auditory mask and then are amplified using the computed insertion gain.Main results. The detection of the attended speech and the HTs are achieved by classifiers that are trained with features extracted from the scalp EEG or the ear EEG signals. The results of evaluating AAD and HT detection show high detection accuracies. The systematic evaluations of the proposed system yield substantial intelligibility and quality improvements for the HI and normal-hearingaudiograms.Significance. The AAD method determines the direction of attention from single-trial EEG signals without access to audio signals of the speakers. The amplification procedure could be adjusted for each subject based on the individual HTs. The present model has the potential to be considered as an important processing tool to personalize the neuro-steered hearing aids.

Dynamic selective auditory attention detection using RNN and reinforcement learning

The cocktail party phenomenon describes the ability of the human brain to focus auditory attention on a particular stimulus while ignoring other acoustic events. Selective auditory attention detection (SAAD) is an important issue in the development of brain-computer interface systems and cocktail party processors. This paper proposes a new dynamic attention detection system to process the temporal evolution of the input signal. The proposed dynamic SAAD is modeled as a sequential decision-making problem, which is solved by recurrent neural network (RNN) and reinforcement learning methods of Q-learning and deep Q-learning. Among different dynamic learning approaches, the evaluation results show that the deep Q-learning approach with RNN as agent provides the highest classification accuracy (94.2%) with the least detection delay. The proposed SAAD system is advantageous, in the sense that the detection of attention is performed dynamically for the sequential inputs. Also, the system has the potential to be used in scenarios, where the attention of the listener might be switched in time in the presence of various acoustic events.

On the estimate of music appraisal from surface EEG: a dynamic-network approach based on cross-sensor PAC measurements

Objective.The aesthetic evaluation of music is strongly dependent on the listener and reflects manifold brain processes that go well beyond the perception of incident sound. Being a high-level cognitive reaction, it is difficult to predict merely from the acoustic features of the audio signal and this poses serious challenges to contemporary music recommendation systems. We attempted to decode music appraisal from brain activity, recorded via wearable EEG, during music listening.Approach.To comply with the dynamic nature of music stimuli, cross-frequency coupling measurements were employed in a time-evolving manner to capture the evolving interactions between distinct brain-rhythms during music listening. Brain response to music was first represented as a continuous flow of functional couplings referring to both regional and inter-regional brain dynamics and then modelled as an ensemble of time-varying (sub)networks. Dynamic graph centrality measures were derived, next, as the final feature-engineering step and, lastly, a support-vector machine was trained to decode the subjective music appraisal. A carefully designed experimental paradigm provided the labeled brain signals.Main results.Using data from 20 subjects, dynamic programming to tailor the decoder to each subject individually and cross-validation, we demonstrated highly satisfactory performance (MAE= 0.948,R²= 0.63) that can be attributed, mostly, to interactions of left frontal gamma rhythm. In addition, our music-appraisal decoder was also employed in a part of the DEAP dataset with similar success. Finally, even a generic version of the decoder (common for all subjects) was found to perform sufficiently.Significance.A novel brain signal decoding scheme was introduced and validated empirically on suitable experimental data. It requires simple operations and leaves room for real-time implementation. Both the code and the experimental data are publicly available.

Rat Locomotion Detection Based on Brain Functional Directed Connectivity from Implanted Electroencephalography Signals

Previous findings have suggested that the cortex involved in walking control in freely locomotion rats. Moreover, the spectral characteristics of cortical activity showed significant differences in different walking conditions. However, whether brain connectivity presents a significant difference during rats walking under different behavior conditions has yet to be verified. Similarly, whether brain connectivity can be used in locomotion detection remains unknown. To address those concerns, we recorded locomotion and implanted electroencephalography signals in freely moving rats performing two kinds of task conditions (upslope and downslope walking). The Granger causality method was used to determine brain functional directed connectivity in rats during these processes. Machine learning algorithms were then used to categorize the two walking states, based on functional directed connectivity. We found significant differences in brain functional directed connectivity varied between upslope and downslope walking. Moreover, locomotion detection based on brain connectivity achieved the highest accuracy (91.45%), sensitivity (90.93%), specificity (91.3%), and F1-score (91.43%). Specifically, the classification results indicated that connectivity features in the high gamma band contained the most discriminative information with respect to locomotion detection in rats, with the support vector machine classifier exhibiting the most efficient performance. Our study not only suggests that brain functional directed connectivity in rats showed significant differences in various behavioral contexts but also proposed a method for classifying the locomotion states of rat walking, based on brain functional directed connectivity. These findings elucidate the characteristics of neural information interaction between various cortical areas in freely walking rats.

Decoding working memory task condition using magnetoencephalography source level long-range phase coupling patterns

OBJECTIVE

The objective of the study is to identify phase coupling patterns that are shared across subjects via a machine learning approach that utilises source space magnetoencephalography (MEG) phase coupling data from a working memory (WM) task. Indeed, phase coupling of neural oscillations is putatively a key factor for communication between distant brain areas and is therefore crucial in performing cognitive tasks, including WM. Previous studies investigating phase coupling during cognitive tasks have often focused on a few a priori selected brain areas or a specific frequency band, and the need for data-driven approaches has been recognised. Machine learning techniques have emerged as valuable tools for the analysis of neuroimaging data since they catch fine-grained differences in the multivariate signal distribution. Here, we expect that these techniques applied to MEG phase couplings can reveal WM-related processes that are shared across individuals.

APPROACH

We analysed WM data collected as part of the Human Connectome Project. The MEG data were collected while subjects (n = 83) performed N-back WM tasks in two different conditions, namely 2-back (WM condition) and 0-back (control condition). We estimated phase coupling patterns (multivariate phase slope index) for both conditions and for theta, alpha, beta, and gamma bands. The obtained phase coupling data were then used to train a linear support vector machine in order to classify which task condition the subject was performing with an across-subject cross-validation approach. The classification was performed separately based on the data from individual frequency bands and with all bands combined (multiband). Finally, we evaluated the relative importance of the different features (phase couplings) for classification by the means of feature selection probability.

MAIN RESULTS

The WM condition and control condition were successfully classified based on the phase coupling patterns in the theta (62% accuracy) and alpha bands (60% accuracy) separately. Importantly, the multiband classification showed that phase coupling patterns not only in the theta and alpha but also in the gamma bands are related to WM processing, as testified by improvement in classification performance (71%).

SIGNIFICANCE

Our study successfully decoded WM tasks using MEG source space functional connectivity. Our approach, combining across-subject classification and a multidimensional metric recently developed by our group, is able to detect patterns of connectivity that are shared across individuals. In other words, the results are generalisable to new individuals and allow meaningful interpretation of task-relevant phase coupling patterns.

Implementation of an Online Auditory Attention Detection Model with Electroencephalography in a Dichotomous Listening Experiment

Auditory attention detection (AAD) is the tracking of a sound source to which a listener is attending based on neural signals. Despite expectation for the applicability of AAD in real-life, most AAD research has been conducted on recorded electroencephalograms (EEGs), which is far from online implementation. In the present study, we attempted to propose an online AAD model and to implement it on a streaming EEG. The proposed model was devised by introducing a sliding window into the linear decoder model and was simulated using two datasets obtained from separate experiments to evaluate the feasibility. After simulation, the online model was constructed and evaluated based on the streaming EEG of an individual, acquired during a dichotomous listening experiment. Our model was able to detect the transient direction of a participant's attention on the order of one second during the experiment and showed up to 70% average detection accuracy. We expect that the proposed online model could be applied to develop adaptive hearing aids or neurofeedback training for auditory attention and speech perception.

Diagnosis of major depressive disorder using whole-brain effective connectivity networks derived from resting-state functional MRI

OBJECTIVE

It is important to improve identification accuracy for possible early intervention of major depressive disorder (MDD). Recently, effective connectivity (EC), defined as the directed influence of spatially distant brain regions on each other, has been used to find the dysfunctional organization of brain networks in MDD. However, little is known about the ability of whole-brain resting-state EC features in identification of MDD. Here, we employed EC by whole-brain analysis to perform MDD diagnosis.

APPROACH

In this study, we proposed a high-order EC network capturing high-level relationship among multiple brain regions to discriminate 57 patients with MDD from 60 normal controls (NC). In high-order EC networks and traditional low-order EC networks, we utilized the network properties and connection strength for classification. Meanwhile, the support vector machine (SVM) was employed for model training. Generalization of the results was supported by 10-fold cross-validation.

MAIN RESULTS

The classification results showed that the high-order EC network performed better than the low-order EC network in diagnosing MDD, and the integration of these two networks yielded the best classification precision with 95% accuracy, 98.83% sensitivity, and 91% specificity. Furthermore, we found that the abnormal connections of high-order EC in MDD patients involved multiple widely concerned functional subnets, particularly the default mode network and the cerebellar network.

SIGNIFICANCE

The current study indicates whole-brain EC networks, measured by our high-order method, may be promising biomarkers for clinical diagnosis of MDD, and the complementary between high-order and low-order EC will better guide patients to get early interventions as well as treatments.