Predictive processing increases intelligibility of acoustically distorted speech: Behavioral and neural correlates

Neural tracking measures of speech intelligibility: Manipulating intelligibility while keeping acoustics unchanged

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle the effects of intelligibility from underlying acoustical confounds. Here, using magnetoencephalography recordings, we study neural measures of speech intelligibility by manipulating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three-band noise-vocoded, ~20 s duration) are presented twice, but the second presentation is preceded by the original (nondegraded) version of the speech. This intermediate priming, which generates a "pop-out" percept, substantially improves the intelligibility of the second degraded speech passage. We investigate how intelligibility and acoustical structure affect acoustic and linguistic neural representations using multivariate temporal response functions (mTRFs). As expected, behavioral results confirm that perceived speech clarity is improved by priming. mTRFs analysis reveals that auditory (speech envelope and envelope onset) neural representations are not affected by priming but only by the acoustics of the stimuli (bottom-up driven). Critically, our findings suggest that segmentation of sounds into words emerges with better speech intelligibility, and most strongly at the later (~400 ms latency) word processing stage, in prefrontal cortex, in line with engagement of top-down mechanisms associated with priming. Taken together, our results show that word representations may provide some objective measures of speech comprehension.

Neural Tracking Measures of Speech Intelligibility: Manipulating Intelligibility while Keeping Acoustics Unchanged

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle effects of intelligibility from underlying acoustical confounds. Here, using magnetoencephalography (MEG) recordings, we study neural measures of speech intelligibility by manipulating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three-band noise vocoded, ~20 s duration) are presented twice, but the second presentation is preceded by the original (non-degraded) version of the speech. This intermediate priming, which generates a 'pop-out' percept, substantially improves the intelligibility of the second degraded speech passage. We investigate how intelligibility and acoustical structure affects acoustic and linguistic neural representations using multivariate Temporal Response Functions (mTRFs). As expected, behavioral results confirm that perceived speech clarity is improved by priming. TRF analysis reveals that auditory (speech envelope and envelope onset) neural representations are not affected by priming, but only by the acoustics of the stimuli (bottom-up driven). Critically, our findings suggest that segmentation of sounds into words emerges with better speech intelligibility, and most strongly at the later (~400 ms latency) word processing stage, in prefrontal cortex (PFC), in line with engagement of top-down mechanisms associated with priming. Taken together, our results show that word representations may provide some objective measures of speech comprehension.

Multivariate fMRI responses in superior temporal cortex predict visual contributions to, and individual differences in, the intelligibility of noisy speech

Humans have the unique ability to decode the rapid stream of language elements that constitute speech, even when it is contaminated by noise. Two reliable observations about noisy speech perception are that seeing the face of the talker improves intelligibility and the existence of individual differences in the ability to perceive noisy speech. We introduce a multivariate BOLD fMRI measure that explains both observations. In two independent fMRI studies, clear and noisy speech was presented in visual, auditory and audiovisual formats to thirty-seven participants who rated intelligibility. An event-related design was used to sort noisy speech trials by their intelligibility. Individual-differences multidimensional scaling was applied to fMRI response patterns in superior temporal cortex and the dissimilarity between responses to clear speech and noisy (but intelligible) speech was measured. Neural dissimilarity was less for audiovisual speech than auditory-only speech, corresponding to the greater intelligibility of noisy audiovisual speech. Dissimilarity was less in participants with better noisy speech perception, corresponding to individual differences. These relationships held for both single word and entire sentence stimuli, suggesting that they were driven by intelligibility rather than the specific stimuli tested. A neural measure of perceptual intelligibility may aid in the development of strategies for helping those with impaired speech perception.

Increased top-down semantic processing in natural speech linked to better reading in dyslexia

Early research proposed that individuals with developmental dyslexia use contextual information to facilitate lexical access and compensate for phonological deficits. Yet at present there is no corroborating neuro-cognitive evidence. We explored this with a novel combination of magnetoencephalography (MEG), neural encoding and grey matter volume analyses. We analysed MEG data from 41 adult native Spanish speakers (14 with dyslexic symptoms) who passively listened to naturalistic sentences. We used multivariate Temporal Response Function analysis to capture online cortical tracking of both auditory (speech envelope) and contextual information. To compute contextual information tracking we used word-level Semantic Surprisal derived using a Transformer Neural Network language model. We related online information tracking to participants' reading scores and grey matter volumes within the reading-linked cortical network. We found that right hemisphere envelope tracking was related to better phonological decoding (pseudoword reading) for both groups, with dyslexic readers performing worse overall at this task. Consistently, grey matter volume in the superior temporal and bilateral inferior frontal areas increased with better envelope tracking abilities. Critically, for dyslexic readers only, stronger Semantic Surprisal tracking in the right hemisphere was related to better word reading. These findings further support the notion of a speech envelope tracking deficit in dyslexia and provide novel evidence for top-down semantic compensatory mechanisms.

Breathing control, brain, and bodily self-consciousness: Toward immersive digiceuticals to alleviate respiratory suffering

Breathing is peculiar among autonomic functions through several characteristics. It generates a very rich afferent traffic from an array of structures belonging to the respiratory system to various areas of the brain. It is intimately associated with bodily movements. It bears particular relationships with consciousness as its efferent motor control can be automatic or voluntary. In this review within the scope of "respiratory neurophysiology" or "respiratory neuroscience", we describe the physiological organisation of breathing control. We then review findings linking breathing and bodily self-consciousness through respiratory manipulations using virtual reality (VR). After discussing the currently admitted neurophysiological model for dyspnea, as well as a new Bayesian model applied to breathing control, we propose that visuo-respiratory paradigms -as developed in cognitive neuroscience- will foster insights into some of the basic mechanisms of the human respiratory system and will also lead to the development of immersive VR-based digital health tools (i.e. digiceuticals).

Semantic Predictability Facilitates Comprehension of Degraded Speech in a Graded Manner

Previous studies have shown that at moderate levels of spectral degradation, semantic predictability facilitates language comprehension. It is argued that when speech is degraded, listeners have narrowed expectations about the sentence endings; i.e., semantic prediction may be limited to only most highly predictable sentence completions. The main objectives of this study were to (i) examine whether listeners form narrowed expectations or whether they form predictions across a wide range of probable sentence endings, (ii) assess whether the facilitatory effect of semantic predictability is modulated by perceptual adaptation to degraded speech, and (iii) use and establish a sensitive metric for the measurement of language comprehension. For this, we created 360 German Subject-Verb-Object sentences that varied in semantic predictability of a sentence-final target word in a graded manner (high, medium, and low) and levels of spectral degradation (1, 4, 6, and 8 channels noise-vocoding). These sentences were presented auditorily to two groups: One group (n =48) performed a listening task in an unpredictable channel context in which the degraded speech levels were randomized, while the other group (n =50) performed the task in a predictable channel context in which the degraded speech levels were blocked. The results showed that at 4 channels noise-vocoding, response accuracy was higher in high-predictability sentences than in the medium-predictability sentences, which in turn was higher than in the low-predictability sentences. This suggests that, in contrast to the narrowed expectations view, comprehension of moderately degraded speech, ranging from low- to high- including medium-predictability sentences, is facilitated in a graded manner; listeners probabilistically preactivate upcoming words from a wide range of semantic space, not limiting only to highly probable sentence endings. Additionally, in both channel contexts, we did not observe learning effects; i.e., response accuracy did not increase over the course of experiment, and response accuracy was higher in the predictable than in the unpredictable channel context. We speculate from these observations that when there is no trial-by-trial variation of the levels of speech degradation, listeners adapt to speech quality at a long timescale; however, when there is a trial-by-trial variation of the high-level semantic feature (e.g., sentence predictability), listeners do not adapt to low-level perceptual property (e.g., speech quality) at a short timescale.

Difficulties with Speech-in-Noise Perception Related to Fundamental Grouping Processes in Auditory Cortex

In our everyday lives, we are often required to follow a conversation when background noise is present (“speech-in-noise” [SPIN] perception). SPIN perception varies widely—and people who are worse at SPIN perception are also worse at fundamental auditory grouping, as assessed by figure-ground tasks. Here, we examined the cortical processes that link difficulties with SPIN perception to difficulties with figure-ground perception using functional magnetic resonance imaging. We found strong evidence that the earliest stages of the auditory cortical hierarchy (left core and belt areas) are similarly disinhibited when SPIN and figure-ground tasks are more difficult (i.e., at target-to-masker ratios corresponding to 60% rather than 90% performance)—consistent with increased cortical gain at lower levels of the auditory hierarchy. Overall, our results reveal a common neural substrate for these basic (figure-ground) and naturally relevant (SPIN) tasks—which provides a common computational basis for the link between SPIN perception and fundamental auditory grouping.

The Predictive Processing Model of EMDR

Eye Movement Desensitization and Reprocessing Therapy (EMDR) is an effective treatment for Post-traumatic Stress Disorder (PTSD). The Adaptive Information Processing Model (AIP) guides the development and practice of EMDR. The AIP postulates inadequately processed memory as the foundation of PTSD pathology. Predictive Processing postulates that the primary function of the brain is prediction that serves to anticipate the next moment of experience in order to resist the dissipative force of entropy thus facilitating continued survival. Memory is the primary substrate of prediction, and is optimized by an ongoing process of precision weighted prediction error minimization that refines prediction by updating the memories on which it is based. The Predictive Processing model of EMDR postulates that EMDR facilitates the predictive processing of traumatic memory by overcoming the bias against exploration and evidence accumulation. The EMDR protocol brings the traumatic memory into an active state of re-experiencing. Defensive responding and/or low sensory precision preclude evidence accumulation to test the predictions of the traumatic memory in the present. Sets of therapist guided eye movements repeatedly challenge the bias against evidence accumulation and compel sensory sampling of the benign present. Eye movements reset the theta rhythm organizing the flow of information through the brain, facilitating the deployment of both overt and covert attention, and the mnemonic search for associations. Sampling of sensation does not support the predictions of the traumatic memory resulting in prediction error that the brain then attempts to minimize. The net result is a restoration of the integrity of the rhythmic deployment of attention, a recalibration of sensory precision, and the updating (reconsolidation) of the traumatic memory. Thus one prediction of the model is a decrease in Attention Bias Variability, a core dysfunction in PTSD, following successful treatment with EMDR.

Neural networks for sentence comprehension and production: An ALE-based meta-analysis of neuroimaging studies

Comprehending and producing sentences is a complex endeavor requiring the coordinated activity of multiple brain regions. We examined three issues related to the brain networks underlying sentence comprehension and production in healthy individuals: First, which regions are recruited for sentence comprehension and sentence production? Second, are there differences for auditory sentence comprehension vs. visual sentence comprehension? Third, which regions are specifically recruited for the comprehension of syntactically complex sentences? Results from activation likelihood estimation (ALE) analyses (from 45 studies) implicated a sentence comprehension network occupying bilateral frontal and temporal lobe regions. Regions implicated in production (from 15 studies) overlapped with the set of regions associated with sentence comprehension in the left hemisphere, but did not include inferior frontal cortex, and did not extend to the right hemisphere. Modality differences between auditory and visual sentence comprehension were found principally in the temporal lobes. Results from the analysis of complex syntax (from 37 studies) showed engagement of left inferior frontal and posterior temporal regions, as well as the right insula. The involvement of the right hemisphere in the comprehension of these structures has potentially important implications for language treatment and recovery in individuals with agrammatic aphasia following left hemisphere brain damage.

Predictive processing increases intelligibility of acoustically distorted speech: Behavioral and neural correlates

INTRODUCTION

We examined which brain areas are involved in the comprehension of acoustically distorted speech using an experimental paradigm where the same distorted sentence can be perceived at different levels of intelligibility. This change in intelligibility occurs via a single intervening presentation of the intact version of the sentence, and the effect lasts at least on the order of minutes. Since the acoustic structure of the distorted stimulus is kept fixed and only intelligibility is varied, this allows one to study brain activity related to speech comprehension specifically.

METHODS

In a functional magnetic resonance imaging (fMRI) experiment, a stimulus set contained a block of six distorted sentences. This was followed by the intact counterparts of the sentences, after which the sentences were presented in distorted form again. A total of 18 such sets were presented to 20 human subjects.

RESULTS

The blood oxygenation level dependent (BOLD)-responses elicited by the distorted sentences which came after the disambiguating, intact sentences were contrasted with the responses to the sentences presented before disambiguation. This revealed increased activity in the bilateral frontal pole, the dorsal anterior cingulate/paracingulate cortex, and the right frontal operculum. Decreased BOLD responses were observed in the posterior insula, Heschl's gyrus, and the posterior superior temporal sulcus.

CONCLUSIONS

The brain areas that showed BOLD-enhancement for increased sentence comprehension have been associated with executive functions and with the mapping of incoming sensory information to representations stored in episodic memory. Thus, the comprehension of acoustically distorted speech may be associated with the engagement of memory-related subsystems. Further, activity in the primary auditory cortex was modulated by prior experience, possibly in a predictive coding framework. Our results suggest that memory biases the perception of ambiguous sensory information toward interpretations that have the highest probability to be correct based on previous experience.