Audio-visual speech processing in age-related hearing loss: Stronger integration and increased frontal lobe recruitment

Hearing loss is associated with difficulties in understanding speech, especially under adverse listening conditions. In these situations, seeing the speaker improves speech intelligibility in hearing-impaired participants. On the neuronal level, previous research has shown cross-modal plastic reorganization in the auditory cortex following hearing loss leading to altered processing of auditory, visual and audio-visual information. However, how reduced auditory input effects audio-visual speech perception in hearing-impaired subjects is largely unknown. We here investigated the impact of mild to moderate age-related hearing loss on processing audio-visual speech using functional magnetic resonance imaging. Normal-hearing and hearing-impaired participants performed two audio-visual speech integration tasks: a sentence detection task inside the scanner and the McGurk illusion outside the scanner. Both tasks consisted of congruent and incongruent audio-visual conditions, as well as auditory-only and visual-only conditions. We found a significantly stronger McGurk illusion in the hearing-impaired participants, which indicates stronger audio-visual integration. Neurally, hearing loss was associated with an increased recruitment of frontal brain areas when processing incongruent audio-visual, auditory and also visual speech stimuli, which may reflect the increased effort to perform the task. Hearing loss modulated both the audio-visual integration strength measured with the McGurk illusion and brain activation in frontal areas in the sentence task, showing stronger integration and higher brain activation with increasing hearing loss. Incongruent compared to congruent audio-visual speech revealed an opposite brain activation pattern in left ventral postcentral gyrus in both groups, with higher activation in hearing-impaired participants in the incongruent condition. Our results indicate that already mild to moderate hearing loss impacts audio-visual speech processing accompanied by changes in brain activation particularly involving frontal areas. These changes are modulated by the extent of hearing loss.

The effect of age-related hearing loss and listening effort on resting state connectivity

Age-related hearing loss is associated with a decrease in hearing abilities for high frequencies. This increases not only the difficulty to understand speech but also the experienced listening effort. Task based neuroimaging studies in normal-hearing and hearing-impaired participants show an increased frontal activation during effortful speech perception in the hearing-impaired. Whether the increased effort in everyday listening in hearing-impaired even impacts functional brain connectivity at rest is unknown. Nineteen normal-hearing and nineteen hearing-impaired participants with mild to moderate hearing loss participated in the study. Hearing abilities, listening effort and resting state functional connectivity were assessed. Our results indicate no differences in functional connectivity between hearing-impaired and normal-hearing participants. Increased listening effort, however, was related to significantly decreased functional connectivity between the dorsal attention network and the precuneus and superior parietal lobule as well as between the auditory and the inferior frontal cortex. We conclude that already mild to moderate age-related hearing loss can impact resting state functional connectivity. It is however not the hearing loss itself but the individually perceived listening effort that relates to functional connectivity changes.

Neural Signatures of Working Memory in Age-related Hearing Loss

Age-related hearing loss affects the ability to hear high frequencies and therefore leads to difficulties in understanding speech, particularly under adverse listening conditions. This decrease in hearing can be partly compensated by the recruitment of executive functions, such as working memory. The compensatory effort may, however, lead to a decrease in available neural resources compromising cognitive abilities. We here aim to investigate whether mild to moderate hearing loss impacts prefrontal functions and related executive processes and whether these are related to speech-in-noise perception abilities. Nineteen hard of hearing and nineteen age-matched normal-hearing participants performed a working memory task to drive prefrontal activity, which was gauged with functional magnetic resonance imaging. In addition, speech-in-noise understanding, cognitive flexibility and inhibition control were assessed. Our results showed no differences in frontoparietal activation patterns and working memory performance between normal-hearing and hard of hearing participants. The behavioral assessment of further executive functions, however, provided evidence of lower cognitive flexibility in hard of hearing participants. Cognitive flexibility and hearing abilities further predicted speech-in-noise perception. We conclude that neural and behavioral signatures of working memory are intact in mild to moderate hearing loss. Moreover, cognitive flexibility seems to be closely related to hearing impairment and speech-in-noise perception and should, therefore, be investigated in future studies assessing age-related hearing loss and its implications on prefrontal functions.

The Contribution of Cognitive Factors to Individual Differences in Understanding Noise-Vocoded Speech in Young and Older Adults

Noise-vocoded speech is commonly used to simulate the sensation after cochlear implantation as it consists of spectrally degraded speech. High individual variability exists in learning to understand both noise-vocoded speech and speech perceived through a cochlear implant (CI). This variability is partly ascribed to differing cognitive abilities like working memory, verbal skills or attention. Although clinically highly relevant, up to now, no consensus has been achieved about which cognitive factors exactly predict the intelligibility of speech in noise-vocoded situations in healthy subjects or in patients after cochlear implantation. We aimed to establish a test battery that can be used to predict speech understanding in patients prior to receiving a CI. Young and old healthy listeners completed a noise-vocoded speech test in addition to cognitive tests tapping on verbal memory, working memory, lexicon and retrieval skills as well as cognitive flexibility and attention. Partial-least-squares analysis revealed that six variables were important to significantly predict vocoded-speech performance. These were the ability to perceive visually degraded speech tested by the Text Reception Threshold, vocabulary size assessed with the Multiple Choice Word Test, working memory gauged with the Operation Span Test, verbal learning and recall of the Verbal Learning and Retention Test and task switching abilities tested by the Comprehensive Trail-Making Test. Thus, these cognitive abilities explain individual differences in noise-vocoded speech understanding and should be considered when aiming to predict hearing-aid outcome.

Effects of age-related hearing loss and hearing aid experience on sentence processing

Age-related hearing loss typically affects the hearing of high frequencies in older adults. Such hearing loss influences the processing of spoken language, including higher-level processing such as that of complex sentences. Hearing aids may alleviate some of the speech processing disadvantages associated with hearing loss. However, little is known about the relation between hearing loss, hearing aid use, and their effects on higher-level language processes. This neuroimaging (fMRI) study examined these factors by measuring the comprehension and neural processing of simple and complex spoken sentences in hard-of-hearing older adults (n = 39). Neither hearing loss severity nor hearing aid experience influenced sentence comprehension at the behavioral level. In contrast, hearing loss severity was associated with increased activity in left superior frontal areas and the left anterior insula, but only when processing specific complex sentences (i.e. object-before-subject) compared to simple sentences. Longer hearing aid experience in a sub-set of participants (n = 19) was associated with recruitment of several areas outside of the core speech processing network in the right hemisphere, including the cerebellum, the precentral gyrus, and the cingulate cortex, but only when processing complex sentences. Overall, these results indicate that brain activation for language processing is affected by hearing loss as well as subsequent hearing aid use. Crucially, they show that these effects become apparent through investigation of complex but not simple sentences.

Disruptions of default mode network and precuneus connectivity associated with cognitive dysfunctions in tinnitus

Tinnitus is the perception of a ringing, buzzing or hissing sound "in the ear" without external stimulation. Previous research has demonstrated changes in resting-state functional connectivity in tinnitus, but findings do not overlap and are even contradictory. Furthermore, how altered functional connectivity in tinnitus is related to cognitive abilities is currently unknown. Here we investigated resting-state functional connectivity differences between 20 patients with chronic tinnitus and 20 control participants matched in age, sex and hearing loss. All participants underwent functional magnetic resonance imaging, audiometric and cognitive assessments, and filled in questionnaires targeting anxiety and depression. Significant differences in functional connectivity between tinnitus patients and control participants were not obtained. However, we did find significant associations between cognitive scores and functional coupling of the default mode network and the precuneus with the superior parietal lobule, supramarginal gyrus, and orbitofrontal cortex. Further, tinnitus distress correlated with connectivity between the precuneus and the lateral occipital complex. This is the first study providing evidence for disruptions of default mode network and precuneus coupling that are related to cognitive dysfunctions in tinnitus. The constant attempt to decrease the tinnitus sensation might occupy certain brain resources otherwise available for concurrent cognitive operations.

Audio-visual interaction in visual motion detection: Synchrony versus Asynchrony

OBJECTIVE

Detection and identification of moving targets is of paramount importance in everyday life, even if it is not widely tested in optometric practice, mostly for technical reasons. There are clear indications in the literature that in perception of moving targets, vision and hearing interact, for example in noisy surrounds and in understanding speech. The main aim of visual perception, the ability that optometry aims to optimize, is the identification of objects, from everyday objects to letters, but also the spatial orientation of subjects in natural surrounds. To subserve this aim, corresponding visual and acoustic features from the rich spectrum of signals supplied by natural environments have to be combined.

METHODS

Here, we investigated the influence of an auditory motion stimulus on visual motion detection, both with a concrete (left/right movement) and an abstract auditory motion (increase/decrease of pitch).

RESULTS

We found that incongruent audiovisual stimuli led to significantly inferior detection compared to the visual only condition. Additionally, detection was significantly better in abstract congruent than incongruent trials. For the concrete stimuli the detection threshold was significantly better in asynchronous audiovisual conditions than in the unimodal visual condition.

CONCLUSION

We find a clear but complex pattern of partly synergistic and partly inhibitory audio-visual interactions. It seems that asynchrony plays only a positive role in audiovisual motion while incongruence mostly disturbs in simultaneous abstract configurations but not in concrete configurations. As in speech perception in hearing-impaired patients, patients suffering from visual deficits should be able to benefit from acoustic information.

Musical, visual and cognitive deficits after middle cerebral artery infarction

The perception of music can be impaired after a stroke. This dysfunction is called amusia and amusia patients often also show deficits in visual abilities, language, memory, learning, and attention. The current study investigated whether deficits in music perception are selective for musical input or generalize to other perceptual abilities. Additionally, we tested the hypothesis that deficits in working memory or attention account for impairments in music perception. Twenty stroke patients with small infarctions in the supply area of the middle cerebral artery were investigated with tests for music and visual perception, categorization, neglect, working memory and attention. Two amusia patients with selective deficits in music perception and pronounced lesions were identified. Working memory and attention deficits were highly correlated across the patient group but no correlation with musical abilities was obtained. Lesion analysis revealed that lesions in small areas of the putamen and globus pallidus were connected to a rhythm perception deficit. We conclude that neither a general perceptual deficit nor a minor domain general deficit can account for impairments in the music perception task. But we find support for the modular organization of the music perception network with brain areas specialized for musical functions as musical deficits were not correlated to any other impairment.

Relationship between Memory Load and Listening Demands in Age-Related Hearing Impairment

Age-related hearing loss has been associated with increased recruitment of frontal brain areas during speech perception to compensate for the decline in auditory input. This additional recruitment may bind resources otherwise needed for understanding speech. However, it is unknown how increased demands on listening interact with increasing cognitive demands when processing speech in age-related hearing loss. The current study used a full-sentence working memory task manipulating demands on working memory and listening and studied untreated mild to moderate hard of hearing (n = 20) and normal-hearing age-matched participants (n = 19) with functional MRI. On the behavioral level, we found a significant interaction of memory load and listening condition; this was, however, similar for both groups. Under low, but not high memory load, listening condition significantly influenced task performance. Similarly, under easy but not difficult listening conditions, memory load had a significant effect on task performance. On the neural level, as measured by the BOLD response, we found increased responses under high compared to low memory load conditions in the left supramarginal gyrus, left middle frontal gyrus, and left supplementary motor cortex regardless of hearing ability. Furthermore, we found increased responses in the bilateral superior temporal gyri under easy compared to difficult listening conditions. We found no group differences nor interactions of group with memory load or listening condition. This suggests that memory load and listening condition interacted on a behavioral level, however, only the increased memory load was reflected in increased BOLD responses in frontal and parietal brain regions. Hence, when evaluating listening abilities in elderly participants, memory load should be considered as it might interfere with the assessed performance. We could not find any further evidence that BOLD responses for the different memory and listening conditions are affected by mild to moderate age-related hearing loss.

Neural Mechanisms Underlying the Processing of Complex Sentences: An fMRI Study

Previous research has shown effects of syntactic complexity on sentence processing. In linguistics, syntactic complexity (caused by different word orders) is traditionally explained by distinct linguistic operations. This study investigates whether different complex word orders indeed result in distinct patterns of neural activity, as would be expected when distinct linguistic operations are applied. Twenty-two older adults performed an auditory sentence processing paradigm in German with and without increased cognitive load. The results show that without increased cognitive load, complex sentences show distinct activation patterns compared with less complex, canonical sentences: complex object-initial sentences show increased activity in the left inferior frontal and temporal regions, whereas complex adjunct-initial sentences show increased activity in occipital and right superior frontal regions. Increased cognitive load seems to affect the processing of different sentence structures differently, increasing neural activity for canonical sentences, but leaving complex sentences relatively unaffected. We discuss these results in the context of the idea that linguistic operations required for processing sentence structures with higher levels of complexity involve distinct brain operations.

Increased fiber density of the fornix in patients with chronic tinnitus revealed by diffusion-weighted MRI

Up to 45% of the elderly population suffer from chronic tinnitus - the phantom perception of sound that is often perceived as ringing, whistling, or hissing "in the ear" without external stimulation. Previous research investigated white matter changes in tinnitus patients using diffusion-weighted magnetic resonance imaging (DWI) to assess measures such as fractional anisotropy (a measure of microstructural integrity of fiber tracts) or mean diffusivity (a measure for general water diffusion). However, findings overlap only minimally and are sometimes even contradictory. We here present the first study encompassing higher diffusion data that allow to focus on changes in tissue microstructure, such as number of axons (fiber density) and macroscopic alterations, including axon diameter, and a combination of both. In order to deal with the crossing-fibers problem, we applied a fixel-based analysis using a constrained spherical deconvolution signal modeling approach. We investigated differences between tinnitus patients and control participants as well as how cognitive abilities and tinnitus distress are related to changes in white matter morphology in chronic tinnitus. For that aim, 20 tinnitus patients and 20 control participants, matched in age, sex and whether they had hearing loss or not, underwent DWI, audiometric and cognitive assessments, and filled in questionnaires targeting anxiety and depression. Our results showed increased fiber density in the fornix in tinnitus patients compared to control participants. The observed changes might, reflect compensatory structural alterations related to the processing of negative emotions or maladaptive changes related to the reinforced learning of the chronic tinnitus sensation. Due to the low sample size, the study should be seen as a pilot study that motivates further research to investigate underlying white matter morphology alterations in tinnitus.

When Hearing Does Not Mean Understanding: On the Neural Processing of Syntactically Complex Sentences by Listeners With Hearing Loss

Purpose Adults with mild-to-moderate age-related hearing loss typically exhibit issues with speech understanding, but their processing of syntactically complex sentences is not well understood. We test the hypothesis that listeners with hearing loss' difficulties with comprehension and processing of syntactically complex sentences are due to the processing of degraded input interfering with the successful processing of complex sentences. Method We performed a neuroimaging study with a sentence comprehension task, varying sentence complexity (through subject-object order and verb-arguments order) and cognitive demands (presence or absence of a secondary task) within subjects. Groups of older subjects with hearing loss (n = 20) and age-matched normal-hearing controls (n = 20) were tested. Results The comprehension data show effects of syntactic complexity and hearing ability, with normal-hearing controls outperforming listeners with hearing loss, seemingly more so on syntactically complex sentences. The secondary task did not influence off-line comprehension. The imaging data show effects of group, sentence complexity, and task, with listeners with hearing loss showing decreased activation in typical speech processing areas, such as the inferior frontal gyrus and superior temporal gyrus. No interactions between group, sentence complexity, and task were found in the neuroimaging data. Conclusions The results suggest that listeners with hearing loss process speech differently from their normal-hearing peers, possibly due to the increased demands of processing degraded auditory input. Increased cognitive demands by means of a secondary visual shape processing task influence neural sentence processing, but no evidence was found that it does so in a different way for listeners with hearing loss and normal-hearing listeners.

Sound-encoded faces activate the left fusiform face area in the early blind

Face perception in humans and nonhuman primates is accomplished by a patchwork of specialized cortical regions. How these regions develop has remained controversial. In sighted individuals, facial information is primarily conveyed via the visual modality. Early blind individuals, on the other hand, can recognize shapes using auditory and tactile cues. Here we demonstrate that such individuals can learn to distinguish faces from houses and other shapes by using a sensory substitution device (SSD) presenting schematic faces as sound-encoded stimuli in the auditory modality. Using functional MRI, we then asked whether a face-selective brain region like the fusiform face area (FFA) shows selectivity for faces in the same subjects, and indeed, we found evidence for preferential activation of the left FFA by sound-encoded faces. These results imply that FFA development does not depend on experience with visual faces per se but may instead depend on exposure to the geometry of facial configurations.

Treatment of Age-Related Hearing Loss Alters Audiovisual Integration and Resting-State Functional Connectivity: A Randomized Controlled Pilot Trial

Untreated age-related hearing loss increases audiovisual integration and impacts resting state functional brain connectivity. Further, there is a relation between crossmodal plasticity and audiovisual integration strength in cochlear implant patients. However, it is currently unclear whether amplification of the auditory input by hearing aids influences audiovisual integration and resting state functional brain connectivity. We conducted a randomized controlled pilot study to investigate how the McGurk illusion, a common measure for audiovisual integration, and resting state functional brain connectivity of the auditory cortex are altered by six-month hearing aid use. Thirty-two older participants with slight-to-moderate, symmetric, age-related hearing loss were allocated to a treatment or waiting control group and measured one week before and six months after hearing aid fitting with functional magnetic resonance imaging. Our results showed a statistical trend for an increased McGurk illusion after six months of hearing aid use. We further demonstrated that an increase in McGurk susceptibility is related to a decreased hearing aid benefit for auditory speech intelligibility in noise. No significant interaction between group and time point was obtained in the whole-brain resting state analysis. However, a region of interest (ROI)-to-ROI analysis indicated that hearing aid use of six months was associated with a decrease in resting state functional connectivity between the auditory cortex and the fusiform gyrus and that this decrease was related to an increase of perceived McGurk illusions. Our study, therefore, suggests that even short-term hearing aid use alters audiovisual integration and functional brain connectivity between auditory and visual cortices.