Attention selectively modulates cortical entrainment in different regions of the speech spectrum

Hearing Impaired Participants Improve More Under Envelope-Transcranial Alternating Current Stimulation When Signal to Noise Ratio Is High

An issue commonly expressed by hearing aid users is a difficulty to understand speech in complex hearing scenarios, that is, when speech is presented together with background noise or in situations with multiple speakers. Conventional hearing aids are already designed with these issues in mind, using beamforming to only enhance sound from a specific direction, but these are limited in solving these issues as they can only modulate incoming sound at the cochlear level. However, evidence exists that age-related hearing loss might partially be caused later in the hearing processes due to brain processes slowing down and becoming less efficient. In this study, we tested whether it would be possible to improve the hearing process at the cortical level by improving neural tracking of speech. The speech envelopes of target sentences were transformed into an electrical signal and stimulated onto elderly participants' cortices using transcranial alternating current stimulation (tACS). We compared 2 different signal to noise ratios (SNRs) with 5 different delays between sound presentation and stimulation ranging from 50 ms to 150 ms, and the differences in effects between elderly normal hearing and elderly hearing impaired participants. When the task was performed at a high SNR, hearing impaired participants appeared to gain more from envelope-tACS compared to when the task was performed at a lower SNR. This was not the case for normal hearing participants. Furthermore, a post-hoc analysis of the different time-lags suggest that elderly were significantly better at a stimulation time-lag of 150 ms when the task was presented at a high SNR. In this paper, we outline why these effects are worth exploring further, and what they tell us about the optimal tACS time-lag.

Lacking Effects of Envelope Transcranial Alternating Current Stimulation Indicate the Need to Revise Envelope Transcranial Alternating Current Stimulation Methods

In recent years, several studies have reported beneficial effects of transcranial alternating current stimulation (tACS) in experiments regarding sound and speech perception. A new development in this field is envelope-tACS: The goal of this method is to improve cortical entrainment to the speech signal by stimulating with a waveform based on the speech envelope. One challenge of this stimulation method is timing; the electrical stimulation needs to be phase-aligned with the naturally occurring cortical entrainment to the auditory stimuli. Due to individual differences in anatomy and processing speed, the optimal time-lag between presentation of sound and applying envelope-tACS varies between participants. To better investigate the effects of envelope-tACS, we performed a speech comprehension task with a larger amount of time-lags than previous experiments, as well as an equal amount of sham conditions. No significant difference between optimal stimulation time-lag condition and best sham condition was found. Further investigation of the data revealed a significant difference between the positive and negative half-cycles of the stimulation conditions but not for sham. However, we also found a significant learning effect over the course of the experiment which was of comparable size to the effects of envelope-tACS found in previous auditory tACS studies. In this article, we discuss possible explanations for why our findings did not match up with those of previous studies and the issues that come with researching and developing envelope-tACS.

Comparison of Two-Talker Attention Decoding from EEG with Nonlinear Neural Networks and Linear Methods

Auditory attention decoding (AAD) through a brain-computer interface has had a flowering of developments since it was first introduced by Mesgarani and Chang (2012) using electrocorticograph recordings. AAD has been pursued for its potential application to hearing-aid design in which an attention-guided algorithm selects, from multiple competing acoustic sources, which should be enhanced for the listener and which should be suppressed. Traditionally, researchers have separated the AAD problem into two stages: reconstruction of a representation of the attended audio from neural signals, followed by determining the similarity between the candidate audio streams and the reconstruction. Here, we compare the traditional two-stage approach with a novel neural-network architecture that subsumes the explicit similarity step. We compare this new architecture against linear and non-linear (neural-network) baselines using both wet and dry electroencephalogram (EEG) systems. Our results indicate that the new architecture outperforms the baseline linear stimulus-reconstruction method, improving decoding accuracy from 66% to 81% using wet EEG and from 59% to 87% for dry EEG. Also of note was the finding that the dry EEG system can deliver comparable or even better results than the wet, despite the latter having one third as many EEG channels as the former. The 11-subject, wet-electrode AAD dataset for two competing, co-located talkers, the 11-subject, dry-electrode AAD dataset, and our software are available for further validation, experimentation, and modification.

Bilingualism and language similarity modify the neural mechanisms of selective attention

Learning and using multiple languages places major demands on our neurocognitive system, which can impact the way the brain processes information. Here we investigated how early bilingualism influences the neural mechanisms of auditory selective attention, and whether this is further affected by the typological similarity between languages. We tested the neural encoding of continuous attended speech in early balanced bilinguals of typologically similar (Dutch-English) and dissimilar languages (Spanish-English) and compared them to results from English monolinguals we reported earlier. In a dichotic listening paradigm, participants attended to a narrative in their native language while ignoring different types of interference in the other ear. The results revealed that bilingualism modulates the neural mechanisms of selective attention even in the absence of consistent behavioural differences between monolinguals and bilinguals. They also suggested that typological similarity between languages helps fine-tune this modulation, reflecting life-long experiences with resolving competition between more or less similar candidates. The effects were consistent over the time-course of the narrative and suggest that learning a second language at an early age triggers neuroplastic adaptation of the attentional processing system.

Neural Encoding of Attended Continuous Speech under Different Types of Interference

We examined how attention modulates the neural encoding of continuous speech under different types of interference. In an EEG experiment, participants attended to a narrative in English while ignoring a competing stream in the other ear. Four different types of interference were presented to the unattended ear: a different English narrative, a narrative in a language unknown to the listener (Spanish), a well-matched nonlinguistic acoustic interference (Musical Rain), and no interference. Neural encoding of attended and unattended signals was assessed by calculating cross-correlations between their respective envelopes and the EEG recordings. Findings revealed more robust neural encoding for the attended envelopes compared with the ignored ones. Critically, however, the type of the interfering stream significantly modulated this process, with the fully intelligible distractor (English) causing the strongest encoding of both attended and unattended streams and latest dissociation between them and nonintelligible distractors causing weaker encoding and early dissociation between attended and unattended streams. The results were consistent over the time course of the spoken narrative. These findings suggest that attended and unattended information can be differentiated at different depths of processing analysis, with the locus of selective attention determined by the nature of the competing stream. They provide strong support to flexible accounts of auditory selective attention.

Transcranial alternating current stimulation with speech envelopes modulates speech comprehension

Cortical entrainment of the auditory cortex to the broadband temporal envelope of a speech signal is crucial for speech comprehension. Entrainment results in phases of high and low neural excitability, which structure and decode the incoming speech signal. Entrainment to speech is strongest in the theta frequency range (4-8 Hz), the average frequency of the speech envelope. If a speech signal is degraded, entrainment to the speech envelope is weaker and speech intelligibility declines. Besides perceptually evoked cortical entrainment, transcranial alternating current stimulation (tACS) entrains neural oscillations by applying an electric signal to the brain. Accordingly, tACS-induced entrainment in auditory cortex has been shown to improve auditory perception. The aim of the current study was to modulate speech intelligibility externally by means of tACS such that the electric current corresponds to the envelope of the presented speech stream (i.e., envelope-tACS). Participants performed the Oldenburg sentence test with sentences presented in noise in combination with envelope-tACS. Critically, tACS was induced at time lags of 0-250 ms in 50-ms steps relative to sentence onset (auditory stimuli were simultaneous to or preceded tACS). We performed single-subject sinusoidal, linear, and quadratic fits to the sentence comprehension performance across the time lags. We could show that the sinusoidal fit described the modulation of sentence comprehension best. Importantly, the average frequency of the sinusoidal fit was 5.12 Hz, corresponding to the peaks of the amplitude spectrum of the stimulated envelopes. This finding was supported by a significant 5-Hz peak in the average power spectrum of individual performance time series. Altogether, envelope-tACS modulates intelligibility of speech in noise, presumably by enhancing and disrupting (time lag with in- or out-of-phase stimulation, respectively) cortical entrainment to the speech envelope in auditory cortex.

The effect of prior knowledge and intelligibility on the cortical entrainment response to speech

It has been suggested that cortical entrainment plays an important role in speech perception by helping to parse the acoustic stimulus into discrete linguistic units. However, the question of whether the entrainment response to speech depends on the intelligibility of the stimulus remains open. Studies addressing this question of intelligibility have, for the most part, significantly distorted the acoustic properties of the stimulus to degrade the intelligibility of the speech stimulus, making it difficult to compare across "intelligible" and "unintelligible" conditions. To avoid these acoustic confounds, we used priming to manipulate the intelligibility of vocoded speech. We used EEG to measure the entrainment response to vocoded target sentences that are preceded by natural speech (nonvocoded) prime sentences that are either valid (match the target) or invalid (do not match the target). For unintelligible speech, valid primes have the effect of restoring intelligibility. We compared the effect of priming on the entrainment response for both 3-channel (unintelligible) and 16-channel (intelligible) speech. We observed a main effect of priming, suggesting that the entrainment response depends on prior knowledge, but not a main effect of vocoding (16 channels vs. 3 channels). Furthermore, we found no difference in the effect of priming on the entrainment response to 3-channel and 16-channel vocoded speech, suggesting that for vocoded speech, entrainment response does not depend on intelligibility.NEW & NOTEWORTHY Neural oscillations have been implicated in the parsing of speech into discrete, hierarchically organized units. Our data suggest that these oscillations track the acoustic envelope rather than more abstract linguistic properties of the speech stimulus. Our data also suggest that prior experience with the stimulus allows these oscillations to better track the stimulus envelope.

Robustness of speech intelligibility at moderate levels of spectral degradation

The current study investigated how amplitude and phase information differentially contribute to speech intelligibility. Listeners performed a word-identification task after hearing spectrally degraded sentences. Each stimulus was degraded by first dividing it into segments, then the amplitude and phase components of each segment were decorrelated independently to various degrees relative to those of the original segment. Segments were then concatenated into their original sequence to present to the listener. We used three segment lengths: 30 ms (phoneme length), 250 ms (syllable length), and full sentence (non-segmented). We found that for intermediate spectral correlation values, segment length is generally inconsequential to intelligibility. Overall, intelligibility was more adversely affected by phase-spectrum decorrelation than by amplitude-spectrum decorrelation. If the phase information was left intact, decorrelating the amplitude spectrum to intermediate values had no effect on intelligibility. If the amplitude information was left intact, decorrelating the phase spectrum to intermediate values significantly degraded intelligibility. Some exceptions to this rule are described. These results delineate the range of amplitude- and phase-spectrum correlations necessary for speech processing and its dependency on the temporal window of analysis (phoneme or syllable length). Results further point to the robustness of speech information in environments that acoustically degrade cues to intelligibility (e.g., reverberant or noisy environments).

Characterization of neural entrainment to speech with and without slow spectral energy fluctuations in laminar recordings in monkey A1

Neural entrainment, the alignment between neural oscillations and rhythmic stimulation, is omnipresent in current theories of speech processing - nevertheless, the underlying neural mechanisms are still largely unknown. Here, we hypothesized that laminar recordings in non-human primates provide us with important insight into these mechanisms, in particular with respect to processing in cortical layers. We presented one monkey with human everyday speech sounds and recorded neural (as current-source density, CSD) oscillations in primary auditory cortex (A1). We observed that the high-excitability phase of neural oscillations was only aligned with those spectral components of speech the recording site was tuned to; the opposite, low-excitability phase was aligned with other spectral components. As low- and high-frequency components in speech alternate, this finding might reflect a particularly efficient way of stimulus processing that includes the preparation of the relevant neuronal populations to the upcoming input. Moreover, presenting speech/noise sounds without systematic fluctuations in amplitude and spectral content and their time-reversed versions, we found significant entrainment in all conditions and cortical layers. When compared with everyday speech, the entrainment in the speech/noise conditions was characterized by a change in the phase relation between neural signal and stimulus and the low-frequency neural phase was dominantly coupled to activity in a lower gamma-band. These results show that neural entrainment in response to speech without slow fluctuations in spectral energy includes a process with specific characteristics that is presumably preserved across species.