Brain electrical dynamics in speech segmentation depends upon prior experience with the language

A listening advantage for native speech is reflected by attention-related activity in auditory cortex

The listening advantage for native speech is well known, but the neural basis of the effect remains unknown. Here we test the hypothesis that attentional enhancement in auditory cortex is stronger for native speech, using magnetoencephalography. Chinese and German speech stimuli were recorded by a bilingual speaker and combined into a two-stream, cocktail-party scene, with consistent and inconsistent language combinations. A group of native speakers of Chinese and a group of native speakers of German performed a detection task in the cued target stream. Results show that attention enhances negative-going activity in the temporal response function deconvoluted from the speech envelope. This activity is stronger when the target stream is in the native compared to the non-native language, and for inconsistent compared to consistent language stimuli. We interpret the findings to show that the stronger activity for native speech could be related to better top-down prediction of the native speech streams.

The effect of familiarity on neural tracking of music stimuli is modulated by mind wandering

One way to investigate the cortical tracking of continuous auditory stimuli is to use the stimulus reconstruction approach. However, the cognitive and behavioral factors impacting this cortical representation remain largely overlooked. Two possible candidates are familiarity with the stimulus and the ability to resist internal distractions. To explore the possible impacts of these two factors on the cortical representation of natural music stimuli, forty-one participants listened to monodic natural music stimuli while we recorded their neural activity. Using the stimulus reconstruction approach and linear mixed models, we found that familiarity positively impacted the reconstruction accuracy of music stimuli and that this effect of familiarity was modulated by mind wandering.

Human EEG and Recurrent Neural Networks Exhibit Common Temporal Dynamics During Speech Recognition

Recent deep-learning artificial neural networks have shown remarkable success in recognizing natural human speech, however the reasons for their success are not entirely understood. Success of these methods might be because state-of-the-art networks use recurrent layers or dilated convolutional layers that enable the network to use a time-dependent feature space. The importance of time-dependent features in human cortical mechanisms of speech perception, measured by electroencephalography (EEG) and magnetoencephalography (MEG), have also been of particular recent interest. It is possible that recurrent neural networks (RNNs) achieve their success by emulating aspects of cortical dynamics, albeit through very different computational mechanisms. In that case, we should observe commonalities in the temporal dynamics of deep-learning models, particularly in recurrent layers, and brain electrical activity (EEG) during speech perception. We explored this prediction by presenting the same sentences to both human listeners and the Deep Speech RNN and considered the temporal dynamics of the EEG and RNN units for identical sentences. We tested whether the recently discovered phenomenon of envelope phase tracking in the human EEG is also evident in RNN hidden layers. We furthermore predicted that the clustering of dissimilarity between model representations of pairs of stimuli would be similar in both RNN and EEG dynamics. We found that the dynamics of both the recurrent layer of the network and human EEG signals exhibit envelope phase tracking with similar time lags. We also computed the representational distance matrices (RDMs) of brain and network responses to speech stimuli. The model RDMs became more similar to the brain RDM when going from early network layers to later ones, and eventually peaked at the recurrent layer. These results suggest that the Deep Speech RNN captures a representation of temporal features of speech in a manner similar to human brain.