Cross-modal plasticity underpins language recovery after cochlear implantation

Impact of ASL Exposure on Spoken Phonemic Discrimination in Adult CI Users: A Functional Near-Infrared Spectroscopy Study

We examined the impact of exposure to a signed language (American Sign Language, or ASL) at different ages on the neural systems that support spoken language phonemic discrimination in deaf individuals with cochlear implants (CIs). Deaf CI users (N = 18, age = 18-24 yrs) who were exposed to a signed language at different ages and hearing individuals (N = 18, age = 18-21 yrs) completed a phonemic discrimination task in a spoken native (English) and non-native (Hindi) language while undergoing functional near-infrared spectroscopy neuroimaging. Behaviorally, deaf CI users who received a CI early versus later in life showed better English phonemic discrimination, albeit phonemic discrimination was poor relative to hearing individuals. Importantly, the age of exposure to ASL was not related to phonemic discrimination. Neurally, early-life language exposure, irrespective of modality, was associated with greater neural activation of left-hemisphere language areas critically involved in phonological processing during the phonemic discrimination task in deaf CI users. In particular, early exposure to ASL was associated with increased activation in the left hemisphere's classic language regions for native versus non-native language phonemic contrasts for deaf CI users who received a CI later in life. For deaf CI users who received a CI early in life, the age of exposure to ASL was not related to neural activation during phonemic discrimination. Together, the findings suggest that early signed language exposure does not negatively impact spoken language processing in deaf CI users, but may instead potentially offset the negative effects of language deprivation that deaf children without any signed language exposure experience prior to implantation. This empirical evidence aligns with and lends support to recent perspectives regarding the impact of ASL exposure in the context of CI usage.

Changes in visually and auditory attended audiovisual speech processing in cochlear implant users: A longitudinal ERP study

Limited auditory input, whether caused by hearing loss or by electrical stimulation through a cochlear implant (CI), can be compensated by the remaining senses. Specifically for CI users, previous studies reported not only improved visual skills, but also altered cortical processing of unisensory visual and auditory stimuli. However, in multisensory scenarios, it is still unclear how auditory deprivation (before implantation) and electrical hearing experience (after implantation) affect cortical audiovisual speech processing. Here, we present a prospective longitudinal electroencephalography (EEG) study which systematically examined the deprivation- and CI-induced alterations of cortical processing of audiovisual words by comparing event-related potentials (ERPs) in postlingually deafened CI users before and after implantation (five weeks and six months of CI use). A group of matched normal-hearing (NH) listeners served as controls. The participants performed a word-identification task with congruent and incongruent audiovisual words, focusing their attention on either the visual (lip movement) or the auditory speech signal. This allowed us to study the (top-down) attention effect on the (bottom-up) sensory cortical processing of audiovisual speech. When compared to the NH listeners, the CI candidates (before implantation) and the CI users (after implantation) exhibited enhanced lipreading abilities and an altered cortical response at the N1 latency range (90-150 ms) that was characterized by a decreased theta oscillation power (4-8 Hz) and a smaller amplitude in the auditory cortex. After implantation, however, the auditory-cortex response gradually increased and developed a stronger intra-modal connectivity. Nevertheless, task efficiency and activation in the visual cortex was significantly modulated in both groups by focusing attention on the visual as compared to the auditory speech signal, with the NH listeners additionally showing an attention-dependent decrease in beta oscillation power (13-30 Hz). In sum, these results suggest remarkable deprivation effects on audiovisual speech processing in the auditory cortex, which partially reverse after implantation. Although even experienced CI users still show distinct audiovisual speech processing compared to NH listeners, pronounced effects of (top-down) direction of attention on (bottom-up) audiovisual processing can be observed in both groups. However, NH listeners but not CI users appear to show enhanced allocation of cognitive resources in visually as compared to auditory attended audiovisual speech conditions, which supports our behavioural observations of poorer lipreading abilities and reduced visual influence on audition in NH listeners as compared to CI users.

Cross-modal plasticity in children with cochlear implant: converging evidence from EEG and functional near-infrared spectroscopy

Over the first years of life, the brain undergoes substantial organization in response to environmental stimulation. In a silent world, it may promote vision by (i) recruiting resources from the auditory cortex and (ii) making the visual cortex more efficient. It is unclear when such changes occur and how adaptive they are, questions that children with cochlear implants can help address. Here, we examined 7-18 years old children: 50 had cochlear implants, with delayed or age-appropriate language abilities, and 25 had typical hearing and language. High-density electroencephalography and functional near-infrared spectroscopy were used to evaluate cortical responses to a low-level visual task. Evidence for a 'weaker visual cortex response' and 'less synchronized or less inhibitory activity of auditory association areas' in the implanted children with language delays suggests that cross-modal reorganization can be maladaptive and does not necessarily strengthen the dominant visual sense.

Auditory, Visual, and Cognitive Abilities in Normal-Hearing Adults, Hearing Aid Users, and Cochlear Implant Users

OBJECTIVES

Speech understanding is considered a bimodal and bidirectional process, whereby visual information (i.e., speechreading) and also cognitive functions (i.e., top-down processes) are involved. Therefore, the purpose of the present study is twofold: (1) to investigate the auditory (A), visual (V), and cognitive (C) abilities in normal-hearing individuals, hearing aid (HA) users, and cochlear implant (CI) users, and (2) to determine an auditory, visual, cognitive (AVC)-profile providing a comprehensive overview of a person's speech processing abilities, containing a broader variety of factors involved in speech understanding.

DESIGN

Three matched groups of subjects participated in this study: (1) 31 normal-hearing adults (mean age = 58.76), (2) 31 adults with moderate to severe hearing loss using HAs (mean age = 59.31), (3) 31 adults with a severe to profound hearing loss using a CI (mean age = 58.86). The audiological assessments consisted of pure-tone audiometry, speech audiometry in quiet and in noise. For evaluation of the (audio-) visual speech processing abilities, the Test for (Audio) Visual Speech perception was used. The cognitive test battery consisted of the letter-number sequencing task, the letter detection test, and an auditory Stroop test, measuring working memory and processing speed, selective attention, and cognitive flexibility and inhibition, respectively. Differences between the three groups were examined using a one-way analysis of variance or Kruskal-Wallis test, depending on the normality of the variables. Furthermore, a principal component analysis was conducted to determine the AVC-profile.

RESULTS

Normal-hearing individuals scored better for both auditory, and cognitive abilities compared to HA users and CI users, listening in a best aided condition. No significant differences were found for speech understanding in a visual condition, despite a larger audiovisual gain for the HA users and CI users. Furthermore, an AVC-profile was composed based on the different auditory, visual, and cognitive assessments. On the basis of that profile, it is possible to determine one comprehensive score for auditory, visual, and cognitive functioning. In the future, these scores could be used in auditory rehabilitation to determine specific strengths and weaknesses per individual patient for the different abilities related to the process of speech understanding in daily life.

CONCLUSIONS

It is suggested to evaluate individuals with hearing loss from a broader perspective, considering more than only the typical auditory abilities. Also, cognitive and visual abilities are important to take into account to have a more complete overview of the speech understanding abilities in daily life.

Effects of noise and noise reduction on audiovisual speech perception in cochlear implant users: An ERP study

OBJECTIVE

Hearing with a cochlear implant (CI) is difficult in noisy environments, but the use of noise reduction algorithms, specifically ForwardFocus, can improve speech intelligibility. The current event-related potentials (ERP) study examined the electrophysiological correlates of this perceptual improvement.

METHODS

Ten bimodal CI users performed a syllable-identification task in auditory and audiovisual conditions, with syllables presented from the front and stationary noise presented from the sides. Brainstorm was used for spatio-temporal evaluation of ERPs.

RESULTS

CI users revealed an audiovisual benefit as reflected by shorter response times and greater activation in temporal and occipital regions at P2 latency. However, in auditory and audiovisual conditions, background noise hampered speech processing, leading to longer response times and delayed auditory-cortex-activation at N1 latency. Nevertheless, activating ForwardFocus resulted in shorter response times, reduced listening effort and enhanced superior-frontal-cortex-activation at P2 latency, particularly in audiovisual conditions.

CONCLUSIONS

ForwardFocus enhances speech intelligibility in audiovisual speech conditions by potentially allowing the reallocation of attentional resources to relevant auditory speech cues.

SIGNIFICANCE

This study shows for CI users that background noise and ForwardFocus differentially affect spatio-temporal cortical response patterns, both in auditory and audiovisual speech conditions.

Dynamic networks differentiate the language ability of children with cochlear implants

Background

Cochlear implantation (CI) in prelingually deafened children has been shown to be an effective intervention for developing language and reading skill. However, there is a substantial proportion of the children receiving CI who struggle with language and reading. The current study-one of the first to implement electrical source imaging in CI population was designed to identify the neural underpinnings in two groups of CI children with good and poor language and reading skill.

Methods

Data using high density electroencephalography (EEG) under a resting state condition was obtained from 75 children, 50 with CIs having good (HL) or poor language skills (LL) and 25 normal hearing (NH) children. We identified coherent sources using dynamic imaging of coherent sources (DICS) and their effective connectivity computing time-frequency causality estimation based on temporal partial directed coherence (TPDC) in the two CI groups compared to a cohort of age and gender matched NH children.

Findings

Sources with higher coherence amplitude were observed in three frequency bands (alpha, beta and gamma) for the CI groups when compared to normal hearing children. The two groups of CI children with good (HL) and poor (LL) language ability exhibited not only different cortical and subcortical source profiles but also distinct effective connectivity between them. Additionally, a support vector machine (SVM) algorithm using these sources and their connectivity patterns for each CI group across the three frequency bands was able to predict the language and reading scores with high accuracy.

Interpretation

Increased coherence in the CI groups suggest overall that the oscillatory activity in some brain areas become more strongly coupled compared to the NH group. Moreover, the different sources and their connectivity patterns and their association to language and reading skill in both groups, suggest a compensatory adaptation that either facilitated or impeded language and reading development. The neural differences in the two groups of CI children may reflect potential biomarkers for predicting outcome success in CI children.

Evaluation of the Cognitive Function of Adults with Severe Hearing Loss Pre- and Post-Cochlear Implantation Using Verbal Fluency Testing

Hearing loss is a major public health problem with significant evidence correlating it with cognitive performance. Verbal fluency tests are commonly used to assess lexical access. They provide a great deal of information about a subject's cognitive function. The aim of our study was to evaluate phonemic and semantic lexical access abilities in adults with bilateral severe to profound hearing loss and then to re-evaluate a cohort after cochlear implantation. 103 adult subjects underwent phonemic and semantic fluency tests during a cochlear implant candidacy evaluation. Of the total 103 subjects, 43 subjects underwent the same tests at 3 months post-implantation. Our results showed superior performance in phonemic fluency compared to semantic fluency in subjects prior to implantation. Phonemic fluency was positively correlated with semantic fluency. Similarly, individuals with congenital deafness had better semantic lexical access than individuals with acquired deafness. Results at 3 months post-implantation showed an improvement in phonemic fluency. No correlation was found between the evolution of pre- and post-implant fluency and the auditory gain of the cochlear implant, and we found no significant difference between congenital and acquired deafness. Our study shows an improvement in global cognitive function after cochlear implantation without differentiation of the phonemic-semantic pathway.

Plasticity After Hearing Rehabilitation in the Aging Brain

Age-related hearing loss, presbycusis, is an unavoidable sensory degradation, often associated with the progressive decline of cognitive and social functions, and dementia. It is generally considered a natural consequence of the inner-ear deterioration. However, presbycusis arguably conflates a wide array of peripheral and central impairments. Although hearing rehabilitation maintains the integrity and activity of auditory networks and can prevent or revert maladaptive plasticity, the extent of such neural plastic changes in the aging brain is poorly appreciated. By reanalyzing a large-scale dataset of more than 2200 cochlear implant users (CI) and assessing the improvement in speech perception from 6 to 24 months of use, we show that, although rehabilitation improves speech understanding on average, age at implantation only minimally affects speech scores at 6 months but has a pejorative effect at 24 months post implantation. Furthermore, older subjects (>67 years old) were significantly more likely to degrade their performances after 2 years of CI use than the younger patients for each year increase in age. Secondary analysis reveals three possible plasticity trajectories after auditory rehabilitation to account for these disparities: Awakening, reversal of deafness-specific changes; Counteracting, stabilization of additional cognitive impairments; or Decline, independent pejorative processes that hearing rehabilitation cannot prevent. The role of complementary behavioral interventions needs to be considered to potentiate the (re)activation of auditory brain networks.

Exploring neurocognitive factors and brain activation in adult cochlear implant recipients associated with speech perception outcomes-A scoping review

Background

Cochlear implants (CIs) are considered an effective treatment for severe-to-profound sensorineural hearing loss. However, speech perception outcomes are highly variable among adult CI recipients. Top-down neurocognitive factors have been hypothesized to contribute to this variation that is currently only partly explained by biological and audiological factors. Studies investigating this, use varying methods and observe varying outcomes, and their relevance has yet to be evaluated in a review. Gathering and structuring this evidence in this scoping review provides a clear overview of where this research line currently stands, with the aim of guiding future research.

Objective

To understand to which extent different neurocognitive factors influence speech perception in adult CI users with a postlingual onset of hearing loss, by systematically reviewing the literature.

Methods

A systematic scoping review was performed according to the PRISMA guidelines. Studies investigating the influence of one or more neurocognitive factors on speech perception post-implantation were included. Word and sentence perception in quiet and noise were included as speech perception outcome metrics and six key neurocognitive domains, as defined by the DSM-5, were covered during the literature search (Protocol in open science registries: 10.17605/OSF.IO/Z3G7W of searches in June 2020, April 2022).

Results

From 5,668 retrieved articles, 54 articles were included and grouped into three categories using different measures to relate to speech perception outcomes: (1) Nineteen studies investigating brain activation, (2) Thirty-one investigating performance on cognitive tests, and (3) Eighteen investigating linguistic skills.

Conclusion

The use of cognitive functions, recruiting the frontal cortex, the use of visual cues, recruiting the occipital cortex, and the temporal cortex still available for language processing, are beneficial for adult CI users. Cognitive assessments indicate that performance on non-verbal intelligence tasks positively correlated with speech perception outcomes. Performance on auditory or visual working memory, learning, memory and vocabulary tasks were unrelated to speech perception outcomes and performance on the Stroop task not to word perception in quiet. However, there are still many uncertainties regarding the explanation of inconsistent results between papers and more comprehensive studies are needed e.g., including different assessment times, or combining neuroimaging and behavioral measures.

Systematic review registration

https://doi.org/10.17605/OSF.IO/Z3G7W.

Intensive Training of Spatial Hearing Promotes Auditory Abilities of Bilateral Cochlear Implant Adults: A Pilot Study

OBJECTIVE

The aim of this study was to evaluate the feasibility of a virtual reality-based spatial hearing training protocol in bilateral cochlear implant (CI) users and to provide pilot data on the impact of this training on different qualities of hearing.

DESIGN

Twelve bilateral CI adults aged between 19 and 69 followed an intensive 10-week rehabilitation program comprised eight virtual reality training sessions (two per week) interspersed with several evaluation sessions (2 weeks before training started, after four and eight training sessions, and 1 month after the end of training). During each 45-minute training session, participants localized a sound source whose position varied in azimuth and/or in elevation. At the start of each trial, CI users received no information about sound location, but after each response, feedback was given to enable error correction. Participants were divided into two groups: a multisensory feedback group (audiovisual spatial cue) and an unisensory group (visual spatial cue) who only received feedback in a wholly intact sensory modality. Training benefits were measured at each evaluation point using three tests: 3D sound localization in virtual reality, the French Matrix test, and the Speech, Spatial and other Qualities of Hearing questionnaire.

RESULTS

The training was well accepted and all participants attended the whole rehabilitation program. Four training sessions spread across 2 weeks were insufficient to induce significant performance changes, whereas performance on all three tests improved after eight training sessions. Front-back confusions decreased from 32% to 14.1% ( p = 0.017); speech recognition threshold score from 1.5 dB to -0.7 dB signal-to-noise ratio ( p = 0.029) and eight CI users successfully achieved a negative signal-to-noise ratio. One month after the end of structured training, these performance improvements were still present, and quality of life was significantly improved for both self-reports of sound localization (from 5.3 to 6.7, p = 0.015) and speech understanding (from 5.2 to 5.9, p = 0.048).

CONCLUSIONS

This pilot study shows the feasibility and potential clinical relevance of this type of intervention involving a sensorial immersive environment and could pave the way for more systematic rehabilitation programs after cochlear implantation.

Cortical tracking of formant modulations derived from silently presented lip movements and its decline with age

The integration of visual and auditory cues is crucial for successful processing of speech, especially under adverse conditions. Recent reports have shown that when participants watch muted videos of speakers, the phonological information about the acoustic speech envelope, which is associated with but independent from the speakers' lip movements, is tracked by the visual cortex. However, the speech signal also carries richer acoustic details, for example, about the fundamental frequency and the resonant frequencies, whose visuophonological transformation could aid speech processing. Here, we investigated the neural basis of the visuo-phonological transformation processes of these more fine-grained acoustic details and assessed how they change as a function of age. We recorded whole-head magnetoencephalographic (MEG) data while the participants watched silent normal (i.e., natural) and reversed videos of a speaker and paid attention to their lip movements. We found that the visual cortex is able to track the unheard natural modulations of resonant frequencies (or formants) and the pitch (or fundamental frequency) linked to lip movements. Importantly, only the processing of natural unheard formants decreases significantly with age in the visual and also in the cingulate cortex. This is not the case for the processing of the unheard speech envelope, the fundamental frequency, or the purely visual information carried by lip movements. These results show that unheard spectral fine details (along with the unheard acoustic envelope) are transformed from a mere visual to a phonological representation. Aging affects especially the ability to derive spectral dynamics at formant frequencies. As listening in noisy environments should capitalize on the ability to track spectral fine details, our results provide a novel focus on compensatory processes in such challenging situations.

Cochlear implants: Visual evoked potentials study

Cochlear implants (CIs) are a successful alternative in cases with severe-to-profound HL. In these individuals, visual cross-modal re-organization can occur because of hearing loss where the visual cortex will recruit auditory cortical areas for visual processing.

OBJECTIVES

This work is designed to study visual evoked potentials (VEPs) in children fitted with CIs in comparison to normal hearing children.

METHOD

This work included 2 groups of children: Group I included 20 normal hearing children and study group included 25 children fitted with unilateral CIs. All cases were subjected to Thorough otological history. Check up on CIs performance using physical check and Aided sound field examination, ophthalmic examination and Pattern Visual Evoked Potentials (pVEPs).

RESULTS

Both groups showed no significant difference as regard age or sex. And both had normal ophthalmic examinations. Children of the study groups showed satisfactory aided response. As regard pVEPs, the study group showed significant higher P100 amplitude in comparison to the control group.

CONCLUSION

This study showed that deafness could induced cortical organization in the visual cortex and not limited to the auditory cortex only.

A direct comparison of voice pitch processing in acoustic and electric hearing

In single-sided deafness patients fitted with a cochlear implant (CI) in the affected ear and preserved normal hearing in the other ear, acoustic and electric hearing can be directly compared without the need for an external control group. Although poor pitch perception is a crucial limitation when listening through CIs, it remains unclear how exactly the cortical processing of pitch information differs between acoustic and electric hearing. Hence, we separately presented both ears of 20 of these patients with vowel sequences in which the pitch contours were either repetitive or variable, while simultaneously recording functional near-infrared spectroscopy (fNIRS) and EEG data. Overall, the results showed smaller and delayed auditory cortex activity in electric hearing, particularly for the P2 event-related potential component, which appears to reflect the processing of voice pitch information. Both the fNIRS data and EEG source reconstructions furthermore showed that vowel sequences with variable pitch contours evoked additional activity in posterior right auditory cortex in electric but not acoustic hearing. This surprising discrepancy demonstrates, firstly, that the acoustic detail transmitted by CIs is sufficient to distinguish between speech sounds that only vary regarding their pitch information. Secondly, the absence of a condition difference when stimulating the normal-hearing ears suggests a saturation of cortical activity levels following unilateral deafness. Taken together, these results provide strong evidence in favour of using CIs in this patient group.

Cross-modal functional connectivity supports speech understanding in cochlear implant users

Sensory deprivation can lead to cross-modal cortical changes, whereby sensory brain regions deprived of input may be recruited to perform atypical function. Enhanced cross-modal responses to visual stimuli observed in auditory cortex of postlingually deaf cochlear implant (CI) users are hypothesized to reflect increased activation of cortical language regions, but it is unclear if this cross-modal activity is "adaptive" or "mal-adaptive" for speech understanding. To determine if increased activation of language regions is correlated with better speech understanding in CI users, we assessed task-related activation and functional connectivity of auditory and visual cortices to auditory and visual speech and non-speech stimuli in CI users (n = 14) and normal-hearing listeners (n = 17) and used functional near-infrared spectroscopy to measure hemodynamic responses. We used visually presented speech and non-speech to investigate neural processes related to linguistic content and observed that CI users show beneficial cross-modal effects. Specifically, an increase in connectivity between the left auditory and visual cortices-presumed primary sites of cortical language processing-was positively correlated with CI users' abilities to understand speech in background noise. Cross-modal activity in auditory cortex of postlingually deaf CI users may reflect adaptive activity of a distributed, multimodal speech network, recruited to enhance speech understanding.

Side-of-Implantation Effect on Functional Asymmetry in the Auditory Cortex of Single-Sided Deaf Cochlear-Implant Users

Cochlear implants (CIs) allow to restore the hearing function in profoundly deaf individuals. Due to the degradation of the stimulus by CI signal processing, implanted individuals with single-sided deafness (SSD) have the specific challenge that the input highly differs between their ears. The present study compared normal-hearing (NH) listeners (N = 10) and left- and right-ear implanted SSD CI users (N = 10 left, N = 9 right), to evaluate cortical speech processing between CI- and NH-ears and to explore for side-of-implantation effects. The participants performed a two-deviant oddball task, separately with the left and the right ear. Auditory event-related potentials (ERPs) in response to syllables were compared between proficient and non-proficient CI users, as well as between CI and NH ears. The effect of the side of implantation was analysed on the sensor and the source level. CI proficiency could be distinguished based on the ERP amplitudes of the N1 and the P3b. Moreover, syllable processing via the CI ear, when compared to the NH ear, resulted in attenuated and delayed ERPs. In addition, the left-ear implanted SSD CI users revealed an enhanced functional asymmetry in the auditory cortex than right-ear implanted SSD CI users, regardless of whether the syllables were perceived via the CI or the NH ear. Our findings reveal that speech-discrimination proficiency in SSD CI users can be assessed by N1 and P3b ERPs. The results contribute to a better understanding of the rehabilitation success in SSD CI users by showing that cortical speech processing in SSD CI users is affected by CI-related stimulus degradation and experience-related functional changes in the auditory cortex.

The timecourse of multisensory speech processing in unilaterally stimulated cochlear implant users revealed by ERPs

•

Both normal-hearing (NH) and cochlear implant (CI) users show a clear benefit in multisensory speech processing.

•

Group differences in ERP topographies and cortical source activation suggest distinct audiovisual speech processing in CI users when compared to NH listeners.

•

Electrical neuroimaging, including topographic and ERP source analysis, provides a suitable tool to study the timecourse of multisensory speech processing in CI users.

A cochlear implant (CI) is an auditory prosthesis which can partially restore the auditory function in patients with severe to profound hearing loss. However, this bionic device provides only limited auditory information, and CI patients may compensate for this limitation by means of a stronger interaction between the auditory and visual system. To better understand the electrophysiological correlates of audiovisual speech perception, the present study used electroencephalography (EEG) and a redundant target paradigm. Postlingually deafened CI users and normal-hearing (NH) listeners were compared in auditory, visual and audiovisual speech conditions. The behavioural results revealed multisensory integration for both groups, as indicated by shortened response times for the audiovisual as compared to the two unisensory conditions. The analysis of the N1 and P2 event-related potentials (ERPs), including topographic and source analyses, confirmed a multisensory effect for both groups and showed a cortical auditory response which was modulated by the simultaneous processing of the visual stimulus. Nevertheless, the CI users in particular revealed a distinct pattern of N1 topography, pointing to a strong visual impact on auditory speech processing. Apart from these condition effects, the results revealed ERP differences between CI users and NH listeners, not only in N1/P2 ERP topographies, but also in the cortical source configuration. When compared to the NH listeners, the CI users showed an additional activation in the visual cortex at N1 latency, which was positively correlated with CI experience, and a delayed auditory-cortex activation with a reversed, rightward functional lateralisation. In sum, our behavioural and ERP findings demonstrate a clear audiovisual benefit for both groups, and a CI-specific alteration in cortical activation at N1 latency when auditory and visual input is combined. These cortical alterations may reflect a compensatory strategy to overcome the limited CI input, which allows the CI users to improve the lip-reading skills and to approximate the behavioural performance of NH listeners in audiovisual speech conditions. Our results are clinically relevant, as they highlight the importance of assessing the CI outcome not only in auditory-only, but also in audiovisual speech conditions.

Recording EEG in Cochlear Implant Users: Guidelines for Experimental Design and Data Analysis for Optimizing Signal Quality and Minimizing Artifacts

Cochlear implants (CI) are neural prostheses that can restore hearing in individuals with severe to profound hearing loss. Although CIs significantly improve quality of life, clinical outcomes are still highly variable. An important part of this variability is explained by the brain reorganization following cochlear implantation. Therefore, clinicians and researchers are seeking objective measurements to investigate post-implantation brain plasticity. Electroencephalography (EEG) is a promising technique because it is objective, non-invasive, and implant-compatible, but is nonetheless susceptible to massive artifacts generated by the prosthesis's electrical activity. CI artifacts can blur and distort brain responses; thus, it is crucial to develop reliable techniques to remove them from EEG recordings. Despite numerous artifact removal techniques used in previous studies, there is a paucity of documentation and consensus on the optimal EEG procedures to reduce these artifacts. Herein, and through a comprehensive review process, we provide a guideline for designing an EEG-CI experiment minimizing the effect of the artifact. We provide some technical guidance for recording an accurate neural response from CI users and discuss the current challenges in detecting and removing CI-induced artifacts from a recorded signal. The aim of this paper is also to provide recommendations to better appraise and report EEG-CI findings.

Distinct multisensory perceptual processes guide enhanced auditory recognition memory in older cochlear implant users

•

Congruent audio-visual encoding enhances later auditory processing in the elderly.

•

CI users benefit from additional congruent visual information, similar to controls.

•

CI users show distinct neurophysiological processes, compared to controls.

•

CI users show an earlier modulation of event-related topographies, compared to controls.

In naturalistic situations, sounds are often perceived in conjunction with matching visual impressions. For example, we see and hear the neighbor’s dog barking in the garden. Still, there is a good chance that we recognize the neighbor’s dog even when we only hear it barking, but do not see it behind the fence. Previous studies with normal-hearing (NH) listeners have shown that the audio-visual presentation of a perceptual object (like an animal) increases the probability to recognize this object later on, even if the repeated presentation of this object occurs in a purely auditory condition. In patients with a cochlear implant (CI), however, the electrical hearing of sounds is impoverished, and the ability to recognize perceptual objects in auditory conditions is significantly limited. It is currently not well understood whether CI users – as NH listeners – show a multisensory facilitation for auditory recognition. The present study used event-related potentials (ERPs) and a continuous recognition paradigm with auditory and audio-visual stimuli to test the prediction that CI users show a benefit from audio-visual perception. Indeed, the congruent audio-visual context resulted in an improved recognition ability of objects in an auditory-only condition, both in the NH listeners and the CI users. The ERPs revealed a group-specific pattern of voltage topographies and correlations between these ERP maps and the auditory recognition ability, indicating a different processing of congruent audio-visual stimuli in CI users when compared to NH listeners. Taken together, our results point to distinct cortical processing of naturalistic audio-visual objects in CI users and NH listeners, which however allows both groups to improve the recognition ability of these objects in a purely auditory context. Our findings are of relevance for future clinical research since audio-visual perception might also improve the auditory rehabilitation after cochlear implantation.

Neuroplasticity following cochlear implants

The auditory cortex of people with sensorineural hearing loss can be re-afferented using a cochlear implant (CI): a neural prosthesis that bypasses the damaged cells in the cochlea to directly stimulate the auditory nerve. Although CIs are the most successful neural prosthesis to date, some CI users still do not achieve satisfactory outcomes using these devices. To explain variability in outcomes, clinicians and researchers have increasingly focused their attention on neuroscientific investigations that examined how the auditory cortices respond to the electric signals that originate from the CI. This chapter provides an overview of the literature that examined how the auditory cortex changes its functional properties in response to inputs from the CI, in animal models and in humans. We focus first on the basic responses to sounds delivered through electrical hearing and, next, we examine the integrity of two fundamental aspects of the auditory system: tonotopy and processing of binaural cues. When addressing the effects of CIs in humans, we also consider speech-evoked responses. We conclude by discussing to what extent this neuroscientific literature can contribute to clinical practices and help to overcome variability in outcomes.

Use of Functional Near-Infrared Spectroscopy to Predict and Measure Cochlear Implant Outcomes: A Scoping Review

Outcomes following cochlear implantation vary widely for both adults and children, and behavioral tests are currently relied upon to assess this. However, these behavioral tests rely on subjective judgements that can be unreliable, particularly for infants and young children. The addition of an objective test of outcome following cochlear implantation is therefore desirable. The aim of this scoping review was to comprehensively catalogue the evidence for the potential of functional near infrared spectroscopy (fNIRS) to be used as a tool to objectively predict and measure cochlear implant outcomes. A scoping review of the literature was conducted following the PRISMA extension for scoping review framework. Searches were conducted in the MEDLINE, EMBASE, PubMed, CINAHL, SCOPUS, and Web of Science electronic databases, with a hand search conducted in Google Scholar. Key terms relating to near infrared spectroscopy and cochlear implants were used to identify relevant publications. Eight records met the criteria for inclusion. Seven records reported on adult populations, with five records only including post-lingually deaf individuals and two including both pre- and post-lingually deaf individuals. Studies were either longitudinal or cross-sectional, and all studies compared fNIRS measurements with receptive speech outcomes. This review identified and collated key work in this field. The homogeneity of the populations studied so far identifies key gaps for future research, including the use of fNIRS in infants. By mapping the literature on this important topic, this review contributes knowledge towards the improvement of outcomes following cochlear implantation.

Brain plasticity and hearing disorders

Permanently changed sensory stimulation can modify functional connectivity patterns in the healthy brain and in pathology. In the pathology case, these adaptive modifications of the brain are referred to as compensation, and the subsequent configurations of functional connectivity are called compensatory plasticity. The variability and extent of auditory deficits due to the impairments in the hearing system determine the related brain reorganization and rehabilitation. In this review, we consider cross-modal and intra-modal brain plasticity related to bilateral and unilateral hearing loss and their restoration using cochlear implantation. Cross-modal brain plasticity may have both beneficial and detrimental effects on hearing disorders. It has a beneficial effect when it serves to improve a patient's adaptation to the visuo-auditory environment. However, the occupation of the auditory cortex by visual functions may be a negative factor for the restoration of hearing with cochlear implants. In what concerns intra-modal plasticity, the loss of interhemispheric asymmetry in asymmetric hearing loss is deleterious for the auditory spatial localization. Research on brain plasticity in hearing disorders can advance our understanding of brain plasticity and improve the rehabilitation of the patients using prognostic, evidence-based approaches from cognitive neuroscience combined with post-rehabilitation objective biomarkers of this plasticity utilizing neuroimaging.

Mirror Mechanism Behind Visual-Auditory Interaction: Evidence From Event-Related Potentials in Children With Cochlear Implants

The mechanism underlying visual-induced auditory interaction is still under discussion. Here, we provide evidence that the mirror mechanism underlies visual–auditory interactions. In this study, visual stimuli were divided into two major groups—mirror stimuli that were able to activate mirror neurons and non-mirror stimuli that were not able to activate mirror neurons. The two groups were further divided into six subgroups as follows: visual speech-related mirror stimuli, visual speech-irrelevant mirror stimuli, and non-mirror stimuli with four different luminance levels. Participants were 25 children with cochlear implants (CIs) who underwent an event-related potential (ERP) and speech recognition task. The main results were as follows: (1) there were significant differences in P1, N1, and P2 ERPs between mirror stimuli and non-mirror stimuli; (2) these ERP differences between mirror and non-mirror stimuli were partly driven by Brodmann areas 41 and 42 in the superior temporal gyrus; (3) ERP component differences between visual speech-related mirror and non-mirror stimuli were partly driven by Brodmann area 39 (visual speech area), which was not observed when comparing the visual speech-irrelevant stimulus and non-mirror groups; and (4) ERPs evoked by visual speech-related mirror stimuli had more components correlated with speech recognition than ERPs evoked by non-mirror stimuli, while ERPs evoked by speech-irrelevant mirror stimuli were not significantly different to those induced by the non-mirror stimuli. These results indicate the following: (1) mirror and non-mirror stimuli differ in their associated neural activation; (2) the visual–auditory interaction possibly led to ERP differences, as Brodmann areas 41 and 42 constitute the primary auditory cortex; (3) mirror neurons could be responsible for the ERP differences, considering that Brodmann area 39 is associated with processing information about speech-related mirror stimuli; and (4) ERPs evoked by visual speech-related mirror stimuli could better reflect speech recognition ability. These results support the hypothesis that a mirror mechanism underlies visual–auditory interactions.

Supra-normal skills in processing of visuo-auditory prosodic information by cochlear-implanted deaf patients

Cochlear implanted (CI) adults with acquired deafness are known to depend on multisensory integration skills (MSI) for speech comprehension through the fusion of speech reading skills and their deficient auditory perception. But, little is known on how CI patients perceive prosodic information relating to speech content. Our study aimed to identify how CI patients use MSI between visual and auditory information to process paralinguistic prosodic information of multimodal speech and the visual strategies employed. A psychophysics assessment was developed, in which CI patients and hearing controls (NH) had to distinguish between a question and a statement. The controls were separated into two age groups (young and aged-matched) to dissociate any effect of aging. In addition, the oculomotor strategies used when facing a speaker in this prosodic decision task were recorded using an eye-tracking device and compared to controls. This study confirmed that prosodic processing is multisensory but it revealed that CI patients showed significant supra-normal audiovisual integration for prosodic information compared to hearing controls irrespective of age. This study clearly showed that CI patients had a visuo-auditory gain more than 3 times larger than that observed in hearing controls. Furthermore, CI participants performed better in the visuo-auditory situation through a specific oculomotor exploration of the face as they significantly fixate the mouth region more than young NH participants who fixate the eyes, whereas the aged-matched controls presented an intermediate exploration pattern equally reported between the eyes and mouth. To conclude, our study demonstrated that CI patients have supra-normal skills MSI when integrating visual and auditory linguistic prosodic information, and a specific adaptive strategy developed as it participates directly in speech content comprehension.

Visual Influences on Auditory Behavioral, Neural, and Perceptual Processes: A Review

In a naturalistic environment, auditory cues are often accompanied by information from other senses, which can be redundant with or complementary to the auditory information. Although the multisensory interactions derived from this combination of information and that shape auditory function are seen across all sensory modalities, our greatest body of knowledge to date centers on how vision influences audition. In this review, we attempt to capture the state of our understanding at this point in time regarding this topic. Following a general introduction, the review is divided into 5 sections. In the first section, we review the psychophysical evidence in humans regarding vision's influence in audition, making the distinction between vision's ability to enhance versus alter auditory performance and perception. Three examples are then described that serve to highlight vision's ability to modulate auditory processes: spatial ventriloquism, cross-modal dynamic capture, and the McGurk effect. The final part of this section discusses models that have been built based on available psychophysical data and that seek to provide greater mechanistic insights into how vision can impact audition. The second section reviews the extant neuroimaging and far-field imaging work on this topic, with a strong emphasis on the roles of feedforward and feedback processes, on imaging insights into the causal nature of audiovisual interactions, and on the limitations of current imaging-based approaches. These limitations point to a greater need for machine-learning-based decoding approaches toward understanding how auditory representations are shaped by vision. The third section reviews the wealth of neuroanatomical and neurophysiological data from animal models that highlights audiovisual interactions at the neuronal and circuit level in both subcortical and cortical structures. It also speaks to the functional significance of audiovisual interactions for two critically important facets of auditory perception-scene analysis and communication. The fourth section presents current evidence for alterations in audiovisual processes in three clinical conditions: autism, schizophrenia, and sensorineural hearing loss. These changes in audiovisual interactions are postulated to have cascading effects on higher-order domains of dysfunction in these conditions. The final section highlights ongoing work seeking to leverage our knowledge of audiovisual interactions to develop better remediation approaches to these sensory-based disorders, founded in concepts of perceptual plasticity in which vision has been shown to have the capacity to facilitate auditory learning.

Hearing loss is associated with gray matter differences in older adults at risk for and with Alzheimer's disease

Using data from the COMPASS-ND study we investigated associations between hearing loss and hippocampal volume as well as cortical thickness in older adults with subjective cognitive decline (SCD), mild cognitive impairment (MCI), and Alzheimer's dementia (AD). SCD participants with greater pure-tone hearing loss exhibited lower hippocampal volume, but more cortical thickness in the left superior temporal gyrus and right pars opercularis. Greater speech-in-noise reception thresholds were associated with lower cortical thickness bilaterally across much of the cortex in AD. The AD group also showed a trend towards worse speech-in-noise thresholds compared to the SCD group.

Combined Brain-Perfusion SPECT and EEG Measurements Suggest Distinct Strategies for Speech Comprehension in CI Users With Higher and Lower Performance

Cochlear implantation constitutes a successful therapy of inner ear deafness, with the majority of patients showing good outcomes. There is, however, still some unexplained variability in outcomes with a number of cochlear-implant (CI) users, showing major limitations in speech comprehension. The current study used a multimodal diagnostic approach combining single-photon emission computed tomography (SPECT) and electroencephalography (EEG) to examine the mechanisms underlying speech processing in postlingually deafened CI users (N = 21). In one session, the participants performed a speech discrimination task, during which a 96-channel EEG was recorded and the perfusions marker 99mTc-HMPAO was injected intravenously. The SPECT scan was acquired 1.5 h after injection to measure the cortical activity during the speech task. The second session included a SPECT scan after injection without stimulation at rest. Analysis of EEG and SPECT data showed N400 and P600 event-related potentials (ERPs) particularly evoked by semantic violations in the sentences, and enhanced perfusion in a temporo-frontal network during task compared to rest, involving the auditory cortex bilaterally and Broca's area. Moreover, higher performance in testing for word recognition and verbal intelligence strongly correlated to the activation in this network during the speech task. However, comparing CI users with lower and higher speech intelligibility [median split with cutoff + 7.6 dB signal-to-noise ratio (SNR) in the Göttinger sentence test] revealed for CI users with higher performance additional activations of parietal and occipital regions and for those with lower performance stronger activation of superior frontal areas. Furthermore, SPECT activity was tightly coupled with EEG and cognitive abilities, as indicated by correlations between (1) cortical activation and the amplitudes in EEG, N400 (temporal and occipital areas)/P600 (parietal and occipital areas) and (2) between cortical activation in left-sided temporal and bilateral occipital/parietal areas and working memory capacity. These results suggest the recruitment of a temporo-frontal network in CI users during speech processing and a close connection between ERP effects and cortical activation in CI users. The observed differences in speech-evoked cortical activation patterns for CI users with higher and lower speech intelligibility suggest distinct processing strategies during speech rehabilitation with CI.

Temporal representation impairment in developmental dyslexia for unisensory and multisensory stimuli

Dyslexia has been associated with a problem in visual-audio integration mechanisms. Here, we investigate for the first time the contribution of unisensory cues on multisensory audio and visual integration in 32 dyslexic children by modelling results using the Bayesian approach. Non-linguistic stimuli were used. Children performed a temporal task: they had to report whether the middle of three stimuli was closer in time to the first one or to the last one presented. Children with dyslexia, compared with typical children, exhibited poorer unimodal thresholds, requiring greater temporal distance between items for correct judgements, while multisensory thresholds were well predicted by the Bayesian model. This result suggests that the multisensory deficit in dyslexia is due to impaired audio and visual inputs rather than impaired multisensory processing per se. We also observed that poorer temporal skills correlated with lower reading skills in dyslexic children, suggesting that this temporal capability can be linked to reading abilities.

Dorsal-movement and ventral-form regions are functionally connected during visual-speech recognition

Faces convey social information such as emotion and speech. Facial emotion processing is supported via interactions between dorsal-movement and ventral-form visual cortex regions. Here, we explored, for the first time, whether similar dorsal-ventral interactions (assessed via functional connectivity), might also exist for visual-speech processing. We then examined whether altered dorsal-ventral connectivity is observed in adults with high-functioning autism spectrum disorder (ASD), a disorder associated with impaired visual-speech recognition. We acquired functional magnetic resonance imaging (fMRI) data with concurrent eye tracking in pairwise matched control and ASD participants. In both groups, dorsal-movement regions in the visual motion area 5 (V5/MT) and the temporal visual speech area (TVSA) were functionally connected to ventral-form regions (i.e., the occipital face area [OFA] and the fusiform face area [FFA]) during the recognition of visual speech, in contrast to the recognition of face identity. Notably, parts of this functional connectivity were decreased in the ASD group compared to the controls (i.e., right V5/MT-right OFA, left TVSA-left FFA). The results confirmed our hypothesis that functional connectivity between dorsal-movement and ventral-form regions exists during visual-speech processing. Its partial dysfunction in ASD might contribute to difficulties in the recognition of dynamic face information relevant for successful face-to-face communication.

Multisensory Integration as a Window into Orderly and Disrupted Cognition and Communication

During our everyday lives, we are confronted with a vast amount of information from several sensory modalities. This multisensory information needs to be appropriately integrated for us to effectively engage with and learn from our world. Research carried out over the last half century has provided new insights into the way such multisensory processing improves human performance and perception; the neurophysiological foundations of multisensory function; the time course for its development; how multisensory abilities differ in clinical populations; and, most recently, the links between multisensory processing and cognitive abilities. This review summarizes the extant literature on multisensory function in typical and atypical circumstances, discusses the implications of the work carried out to date for theory and research, and points toward next steps for advancing the field.

Central Effects of Cranial Nerve Stimulation

The current literature on peripheral cranial nerve stimulation for the purpose of achieving therapeutic effects via altering brain activity is reviewed. Vagus nerve stimulation, which is approved for use in refractory epilepsy, is the most extensively studied cranial nerve stimulator that has direct impact on the central nervous system. Despite the recognized central effects of peripheral cranial nerve stimulation, the mechanism of action for all indications remains incompletely understood. Further research on both mechanisms and indications of central effects of cranial nerve stimulation has the potential to alleviate burden of disease in a large array of conditions.

Auditory Selective Attention Hindered by Visual Stimulus in Prelingually Deaf Children With Cochlear Implants

OBJECTIVE

To compare the influence of visual distractors on the performance of auditory selective attention between prelingually deaf children with a CI (cochlear implant) and children with normal-hearing.

DESIGN

Twenty-two patients who had a cochlear implant device (10 males and 12 females, aged 6.64 ± 0.99 yrs) and 16 normal-hearing children (6 males and 10 females, aged 6.09 ± 0.51 yrs) were recruited. Half of the auditory stimuli were presented together with visual stimuli, and participants were required to complete an auditory identification task. Reaction times and discriminability (d') for these two groups were recorded and calculated.

RESULTS

The normal-hearing group had shorter mean reaction times than the CI group in detecting auditory targets. With visual distraction, the d' of the normal-hearing group was significantly better than that of CI group (t = 2.649, p = 0.012), while no statistical significance was found between the two groups without visual distraction (t = 0.693, p = 0.493).

CONCLUSION

Enhanced processing of visual stimuli interferes with auditory perception in CI users by occupying the capacity-limited attention.

Effects of Cerebral Blood Flow and Vessel Conditions on Speech Recognition in Patients With Postlingual Adult Cochlear Implant: Predictable Factors for the Efficacy of Cochlear Implant

OBJECTIVES

Cochlear implantation (CI) has been the most successful procedure for restoring hearing in a patient with severe and profound hearing loss. However, possibly owing to the variable brain functions of each patient, its performance and the associated patient satisfaction are widely variable. The authors hypothesize that peripheral and cerebral circulation can be assessed by noninvasive and globally available methods, yielding superior presurgical predictive factors of the performance of CI in adult patients with postlingual hearing loss who are scheduled to undergo CI.

DESIGN

Twenty-two adult patients with cochlear implants for postlingual hearing loss were evaluated using Doppler sonography measurement of the cervical arteries (reflecting cerebral blood flow), flow-mediated dilation (FMD; reflecting the condition of cerebral arteries), and their pre-/post-CI best score on a monosyllabic discrimination test (pre-/post-CI best monosyllabic discrimination [BMD] score). Correlations between post-CI BMD score and the other factors were examined using univariate analysis and stepwise multiple linear regression analysis. The prediction factors were calculated by examining the receiver-operating characteristic curve between post-CI BMD score and the significantly positively correlated factors.

RESULTS

Age and duration of deafness had a moderately negative correlation. The mean velocity of the internal carotid arteries and FMD had a moderate-to-strong positive correlation with the post-CI BMD score in univariate analysis. Stepwise multiple linear regression analysis revealed that only FMD was significantly positively correlated with post-CI BMD score. Analysis of the receiver-operating characteristic curve showed that a FMD cutoff score of 1.8 significantly predicted post-CI BMD score.

CONCLUSIONS

These data suggest that FMD is a convenient, noninvasive, and widely available tool for predicting the efficacy of cochlear implants. An FMD cutoff score of 1.8 could be a good index for determining whether patients will hear well with cochlear implants. It could also be used to predict whether cochlear implants will provide good speech recognition benefits to candidates, even if their speech discrimination is poor. This FMD index could become a useful predictive tool for candidates with poor speech discrimination to determine the efficacy of CI before surgery.

Cortical Speech Processing in Postlingually Deaf Adult Cochlear Implant Users, as Revealed by Functional Near-Infrared Spectroscopy

An experiment was conducted to investigate the feasibility of using functional near-infrared spectroscopy (fNIRS) to image cortical activity in the language areas of cochlear implant (CI) users and to explore the association between the activity and their speech understanding ability. Using fNIRS, 15 experienced CI users and 14 normal-hearing participants were imaged while presented with either visual speech or auditory speech. Brain activation was measured from the prefrontal, temporal, and parietal lobe in both hemispheres, including the language-associated regions. In response to visual speech, the activation levels of CI users in an a priori region of interest (ROI)—the left superior temporal gyrus or sulcus—were negatively correlated with auditory speech understanding. This result suggests that increased cross-modal activity in the auditory cortex is predictive of poor auditory speech understanding. In another two ROIs, in which CI users showed significantly different mean activation levels in response to auditory speech compared with normal-hearing listeners, activation levels were significantly negatively correlated with CI users’ auditory speech understanding. These ROIs were located in the right anterior temporal lobe (including a portion of prefrontal lobe) and the left middle superior temporal lobe. In conclusion, fNIRS successfully revealed activation patterns in CI users associated with their auditory speech understanding.

Recognizing visual speech: Reduced responses in visual-movement regions, but not other speech regions in autism

Speech information inherent in face movements is important for understanding what is said in face-to-face communication. Individuals with autism spectrum disorders (ASD) have difficulties in extracting speech information from face movements, a process called visual-speech recognition. Currently, it is unknown what dysfunctional brain regions or networks underlie the visual-speech recognition deficit in ASD. We conducted a functional magnetic resonance imaging (fMRI) study with concurrent eye tracking to investigate visual-speech recognition in adults diagnosed with high-functioning autism and pairwise matched typically developed controls. Compared to the control group (n = 17), the ASD group (n = 17) showed decreased Blood Oxygenation Level Dependent (BOLD) response during visual-speech recognition in the right visual area 5 (V5/MT) and left temporal visual speech area (TVSA) - brain regions implicated in visual-movement perception. The right V5/MT showed positive correlation with visual-speech task performance in the ASD group, but not in the control group. Psychophysiological interaction analysis (PPI) revealed that functional connectivity between the left TVSA and the bilateral V5/MT and between the right V5/MT and the left IFG was lower in the ASD than in the control group. In contrast, responses in other speech-motor regions and their connectivity were on the neurotypical level. Reduced responses and network connectivity of the visual-movement regions in conjunction with intact speech-related mechanisms indicate that perceptual mechanisms might be at the core of the visual-speech recognition deficit in ASD. Communication deficits in ASD might at least partly stem from atypical sensory processing and not higher-order cognitive processing of socially relevant information.

Anatomical and Functional MRI Changes after One Year of Auditory Rehabilitation with Hearing Aids

Hearing aids (HAs) are an effective strategy for auditory rehabilitation in patients with peripheral hearing deficits. Yet, the neurophysiological mechanisms behind HA use are still unclear. Thus far, most studies have focused on changes in the auditory system, although it is expected that hearing deficits affect a number of cognitive systems, notably speech. In the present study, we used audiometric evaluations in 14 patients with bilateral hearing loss before and after one year of continuous HA use and functional magnetic resonance imaging (fMRI) and cortical thickness analysis in 12 and 10 of them compared with a normal hearing control group. Prior to HA fitting, fMRI activity was found reduced in the auditory and language systems and increased in visual and frontal areas, expanding to multimodal integration cortices, such as the superior temporal gyrus, intraparietal sulcus, and insula. One year after rehabilitation with HA, significant audiometric improvement was observed, especially in free-field Speech Reception Threshold (SRT) test and functional gain, a measure of HA efficiency. HA use increased fMRI activity in the auditory and language cortices and multimodal integration areas. Individual fMRI signal changes from all these areas were positively correlated with individual SRT changes. Before rehabilitation, cortical thickness was increased in parts of the prefrontal cortex, precuneus, fusiform gyrus, and middle temporal gyrus. It was reduced in the insula, supramarginal gyrus, medial temporal gyrus, occipital cortex, posterior cingulate cortex, and claustrum. After HA use, increased cortical thickness was observed in multimodal integration regions, particularly the very caudal end of the superior temporal sulcus, the angular gyrus, and the inferior parietal gyrus/superior temporal gyrus/insula. Our data provide the first evidence that one year of HA use is related to functional and anatomical brain changes, notably in auditory and language systems, extending to multimodal cortices.

Task-dependent modulation of the visual sensory thalamus assists visual-speech recognition

The cerebral cortex modulates early sensory processing via feed-back connections to sensory pathway nuclei. The functions of this top-down modulation for human behavior are poorly understood. Here, we show that top-down modulation of the visual sensory thalamus (the lateral geniculate body, LGN) is involved in visual-speech recognition. In two independent functional magnetic resonance imaging (fMRI) studies, LGN response increased when participants processed fast-varying features of articulatory movements required for visual-speech recognition, as compared to temporally more stable features required for face identification with the same stimulus material. The LGN response during the visual-speech task correlated positively with the visual-speech recognition scores across participants. In addition, the task-dependent modulation was present for speech movements and did not occur for control conditions involving non-speech biological movements. In face-to-face communication, visual speech recognition is used to enhance or even enable understanding what is said. Speech recognition is commonly explained in frameworks focusing on cerebral cortex areas. Our findings suggest that task-dependent modulation at subcortical sensory stages has an important role for communication: Together with similar findings in the auditory modality the findings imply that task-dependent modulation of the sensory thalami is a general mechanism to optimize speech recognition.

Multisensory Integration in Cochlear Implant Recipients

Speech perception is inherently a multisensory process involving integration of auditory and visual cues. Multisensory integration in cochlear implant (CI) recipients is a unique circumstance in that the integration occurs after auditory deprivation and the provision of hearing via the CI. Despite the clear importance of multisensory cues for perception, in general, and for speech intelligibility, specifically, the topic of multisensory perceptual benefits in CI users has only recently begun to emerge as an area of inquiry. We review the research that has been conducted on multisensory integration in CI users to date and suggest a number of areas needing further research. The overall pattern of results indicates that many CI recipients show at least some perceptual gain that can be attributable to multisensory integration. The extent of this gain, however, varies based on a number of factors, including age of implantation and specific task being assessed (e.g., stimulus detection, phoneme perception, word recognition). Although both children and adults with CIs obtain audiovisual benefits for phoneme, word, and sentence stimuli, neither group shows demonstrable gain for suprasegmental feature perception. Additionally, only early-implanted children and the highest performing adults obtain audiovisual integration benefits similar to individuals with normal hearing. Increasing age of implantation in children is associated with poorer gains resultant from audiovisual integration, suggesting a sensitive period in development for the brain networks that subserve these integrative functions, as well as length of auditory experience. This finding highlights the need for early detection of and intervention for hearing loss, not only in terms of auditory perception, but also in terms of the behavioral and perceptual benefits of audiovisual processing. Importantly, patterns of auditory, visual, and audiovisual responses suggest that underlying integrative processes may be fundamentally different between CI users and typical-hearing listeners. Future research, particularly in low-level processing tasks such as signal detection will help to further assess mechanisms of multisensory integration for individuals with hearing loss, both with and without CIs.

Visual Temporal Acuity Is Related to Auditory Speech Perception Abilities in Cochlear Implant Users

OBJECTIVES

Despite significant improvements in speech perception abilities following cochlear implantation, many prelingually deafened cochlear implant (CI) recipients continue to rely heavily on visual information to develop speech and language. Increased reliance on visual cues for understanding spoken language could lead to the development of unique audiovisual integration and visual-only processing abilities in these individuals. Brain imaging studies have demonstrated that good CI performers, as indexed by auditory-only speech perception abilities, have different patterns of visual cortex activation in response to visual and auditory stimuli as compared with poor CI performers. However, no studies have examined whether speech perception performance is related to any type of visual processing abilities following cochlear implantation. The purpose of the present study was to provide a preliminary examination of the relationship between clinical, auditory-only speech perception tests, and visual temporal acuity in prelingually deafened adult CI users. It was hypothesized that prelingually deafened CI users, who exhibit better (i.e., more acute) visual temporal processing abilities would demonstrate better auditory-only speech perception performance than those with poorer visual temporal acuity.

DESIGN

Ten prelingually deafened adult CI users were recruited for this study. Participants completed a visual temporal order judgment task to quantify visual temporal acuity. To assess auditory-only speech perception abilities, participants completed the consonant-nucleus-consonant word recognition test and the AzBio sentence recognition test. Results were analyzed using two-tailed partial Pearson correlations, Spearman's rho correlations, and independent samples t tests.

RESULTS

Visual temporal acuity was significantly correlated with auditory-only word and sentence recognition abilities. In addition, proficient CI users, as assessed via auditory-only speech perception performance, demonstrated significantly better visual temporal acuity than nonproficient CI users.

CONCLUSIONS

These findings provide the first behavioral evidence that visual temporal acuity is related to post implantation CI proficiency as indexed by auditory-only speech perception performance. These preliminary data bring to light the possible future role of visual temporal acuity in predicting CI outcomes before implantation, as well as the possible utility of visual training methods in improving CI outcomes.

Auditory cross-modal reorganization in cochlear implant users indicates audio-visual integration

There is clear evidence for cross-modal cortical reorganization in the auditory system of post-lingually deafened cochlear implant (CI) users. A recent report suggests that moderate sensori-neural hearing loss is already sufficient to initiate corresponding cortical changes. To what extend these changes are deprivation-induced or related to sensory recovery is still debated. Moreover, the influence of cross-modal reorganization on CI benefit is also still unclear. While reorganization during deafness may impede speech recovery, reorganization also has beneficial influences on face recognition and lip-reading. As CI users were observed to show differences in multisensory integration, the question arises if cross-modal reorganization is related to audio-visual integration skills. The current electroencephalography study investigated cortical reorganization in experienced post-lingually deafened CI users (n = 18), untreated mild to moderately hearing impaired individuals (n = 18) and normal hearing controls (n = 17). Cross-modal activation of the auditory cortex by means of EEG source localization in response to human faces and audio-visual integration, quantified with the McGurk illusion, were measured. CI users revealed stronger cross-modal activations compared to age-matched normal hearing individuals. Furthermore, CI users showed a relationship between cross-modal activation and audio-visual integration strength. This may further support a beneficial relationship between cross-modal activation and daily-life communication skills that may not be fully captured by laboratory-based speech perception tests. Interestingly, hearing impaired individuals showed behavioral and neurophysiological results that were numerically between the other two groups, and they showed a moderate relationship between cross-modal activation and the degree of hearing loss. This further supports the notion that auditory deprivation evokes a reorganization of the auditory system even at early stages of hearing loss.

Increased cross-modal functional connectivity in cochlear implant users

Previous studies have reported increased cross-modal auditory and visual cortical activation in cochlear implant (CI) users, suggesting cross-modal reorganization of both visual and auditory cortices in CI users as a consequence of sensory deprivation and restoration. How these processes affect the functional connectivity of the auditory and visual system in CI users is however unknown. We here investigated task-induced intra-modal functional connectivity between hemispheres for both visual and auditory cortices and cross-modal functional connectivity between visual and auditory cortices using functional near infrared spectroscopy in post-lingually deaf CI users and age-matched normal hearing controls. Compared to controls, CI users exhibited decreased intra-modal functional connectivity between hemispheres and increased cross-modal functional connectivity between visual and left auditory cortices for both visual and auditory stimulus processing. Importantly, the difference between cross-modal functional connectivity for visual and for auditory stimuli correlated with speech recognition outcome in CI users. Higher cross-modal connectivity for auditory than for visual stimuli was associated with better speech recognition abilities, pointing to a new pattern of functional reorganization that is related to successful hearing restoration with a CI.

Simultaneous bilateral cochlear implants: Developmental advances do not yet achieve normal cortical processing

BACKGROUND

Simultaneous bilateral cochlear implantation promotes symmetric development of bilateral auditory pathways but binaural hearing remains abnormal. To evaluate whether bilateral cortical processing remains impaired in such children, cortical activity to unilateral and bilateral stimuli was assessed in a unique cohort of 16 children who received bilateral cochlear implants (CIs) simultaneously at 1.97 ± 0.86 years of age and had ~4 years of CI experience, providing the first opportunity to assess electrically driven cortical development in the absence of reorganized asymmetries from sequential implantation.

METHODS

Cortical activity to unilateral and bilateral stimuli was measured using multichannel electro-encephalography. Cortical processing in children with bilateral CIs was compared with click-elicited activity in 13 normal hearing children matched for time-in-sound. Source activity was localized using the Time Restricted, Artefact and Coherence source Suppression (TRACS) beamformer method.

RESULTS

Consistent with dominant crossed auditory pathways, normal P1 activity (~100 ms) was weaker to ipsilateral stimuli relative to contralateral and bilateral stimuli and both auditory cortices preferentially responded to the contralateral ear. Right hemisphere dominance was evident overall. Children with bilateral CIs maintained the expected right dominance but differences from normal included: (i) minimal changes between ipsilateral, contralateral and bilateral stimuli, (ii) weaker than normal contralateral stimulus preference, (iii) symmetric activity to bilateral stimuli, and (iv) increased occipital lobe recruitment during bilateral relative to unilateral stimulation. Between-group contrasts demonstrated lower than normal activity in the inferior parieto-occipital lobe (suggesting deficits in sensory integration) and greater than normal left frontal lobe activity (suggesting increased attention), even during passive listening.

CONCLUSIONS

Together, findings suggest that early simultaneous bilateral cochlear implantation promotes normal-like auditory symmetry but that abnormalities in cortical processing consequent to deafness and/or electrical stimulation through two independent speech processors persist.

Audio-visual temporal perception in children with restored hearing

It is not clear how audio-visual temporal perception develops in children with restored hearing. In this study we measured temporal discrimination thresholds with an audio-visual temporal bisection task in 9 deaf children with restored audition, and 22 typically hearing children. In typically hearing children, audition was more precise than vision, with no gain in multisensory conditions (as previously reported in Gori et al. (2012b)). However, deaf children with restored audition showed similar thresholds for audio and visual thresholds and some evidence of gain in audio-visual temporal multisensory conditions. Interestingly, we found a strong correlation between auditory weighting of multisensory signals and quality of language: patients who gave more weight to audition had better language skills. Similarly, auditory thresholds for the temporal bisection task were also a good predictor of language skills. This result supports the idea that the temporal auditory processing is associated with language development.

Faster phonological processing and right occipito-temporal coupling in deaf adults signal poor cochlear implant outcome

The outcome of adult cochlear implantation is predicted positively by the involvement of visual cortex in speech processing, and negatively by the cross-modal recruitment of the right temporal cortex during and after deafness. How these two neurofunctional predictors concur to modulate cochlear implant (CI) performance remains unclear. In this fMRI study, we explore the joint involvement of occipital and right hemisphere regions in a visual-based phonological task in post-lingual deafness. Intriguingly, we show that some deaf subjects perform faster than controls. This behavioural effect is associated with reorganized connectivity across bilateral visual, right temporal and left inferior frontal cortices, but with poor CI outcome. Conversely, preserved normal-range reaction times are associated with left-lateralized phonological processing and good CI outcome. These results suggest that following deafness, involvement of visual cortex in the context of reorganized right-lateralized phonological processing compromises its availability for audio-visual synergy during adaptation to CI.

Cochlear implants have variable outcomes for adult deafness. Here the authors show that fast responses and specific recruitment of right temporal cortex on a simple visual rhyming task strongly predicts poor implant performance.

Speechreading in Deaf Adults with Cochlear Implants: Evidence for Perceptual Compensation

Previous research has provided evidence for a speechreading advantage in congenitally deaf adults compared to hearing adults. A 'perceptual compensation' account of this finding proposes that prolonged early onset deafness leads to a greater reliance on visual, as opposed to auditory, information when perceiving speech which in turn results in superior visual speech perception skills in deaf adults. In the current study we tested whether previous demonstrations of a speechreading advantage for profoundly congenitally deaf adults with hearing aids, or no amplificiation, were also apparent in adults with the same deafness profile but who have experienced greater access to the auditory elements of speech via a cochlear implant (CI). We also tested the prediction that, in line with the perceptual compensation account, receiving a CI at a later age is associated with superior speechreading skills due to later implanted individuals having experienced greater dependence on visual speech information. We designed a speechreading task in which participants viewed silent videos of 123 single words spoken by a model and were required to indicate which word they thought had been said via a free text response. We compared congenitally deaf adults who had received CIs in childhood or adolescence (N = 15) with a comparison group of hearing adults (N = 15) matched on age and education level. The adults with CI showed significantly better scores on the speechreading task than the hearing comparison group. Furthermore, within the group of adults with CI, there was a significant positive correlation between age at implantation and speechreading performance; earlier implantation was associated with lower speechreading scores. These results are both consistent with the hypothesis of perceptual compensation in the domain of speech perception, indicating that more prolonged dependence on visual speech information in speech perception may lead to improvements in the perception of visual speech. In addition our study provides metrics of the 'speechreadability' of 123 words produced in British English: one derived from hearing adults (N = 61) and one from deaf adults with CI (N = 15). Evidence for the validity of these 'speechreadability' metrics come from correlations with visual lexical competition data.

When Hearing Is Tricky: Speech Processing Strategies in Prelingually Deafened Children and Adolescents with Cochlear Implants Having Good and Poor Speech Performance

Cochlear implants provide individuals who are deaf with access to speech. Although substantial advancements have been made by novel technologies, there still is high variability in language development during childhood, depending on adaptation and neural plasticity. These factors have often been investigated in the auditory domain, with the mismatch negativity as an index for sensory and phonological processing. Several studies have demonstrated that the MMN is an electrophysiological correlate for hearing improvement with cochlear implants. In this study, two groups of cochlear implant users, both with very good basic hearing abilities but with non-overlapping speech performance (very good or very poor speech performance), were matched according to device experience and age at implantation. We tested the perception of phonemes in the context of specific other phonemes from which they were very hard to discriminate (e.g., the vowels in /bu/ vs. /bo/). The most difficult pair was individually determined for each participant. Using behavioral measures, both cochlear implants groups performed worse than matched controls, and the good performers performed better than the poor performers. Cochlear implant groups and controls did not differ during time intervals typically used for the mismatch negativity, but earlier: source analyses revealed increased activity in the region of the right supramarginal gyrus (220-260 ms) in good performers. Poor performers showed increased activity in the left occipital cortex (220-290 ms), which may be an index for cross-modal perception. The time course and the neural generators differ from data from our earlier studies, in which the same phonemes were assessed in an easy-to-discriminate context. The results demonstrate that the groups used different language processing strategies, depending on the success of language development and the particular language context. Overall, our data emphasize the role of neural plasticity and use of adaptive strategies for successful language development with cochlear implants.