Neural correlates of visual stimulus encoding and verbal working memory differ between cochlear implant users and normal-hearing controls

Changes in visually and auditory attended audiovisual speech processing in cochlear implant users: A longitudinal ERP study

Limited auditory input, whether caused by hearing loss or by electrical stimulation through a cochlear implant (CI), can be compensated by the remaining senses. Specifically for CI users, previous studies reported not only improved visual skills, but also altered cortical processing of unisensory visual and auditory stimuli. However, in multisensory scenarios, it is still unclear how auditory deprivation (before implantation) and electrical hearing experience (after implantation) affect cortical audiovisual speech processing. Here, we present a prospective longitudinal electroencephalography (EEG) study which systematically examined the deprivation- and CI-induced alterations of cortical processing of audiovisual words by comparing event-related potentials (ERPs) in postlingually deafened CI users before and after implantation (five weeks and six months of CI use). A group of matched normal-hearing (NH) listeners served as controls. The participants performed a word-identification task with congruent and incongruent audiovisual words, focusing their attention on either the visual (lip movement) or the auditory speech signal. This allowed us to study the (top-down) attention effect on the (bottom-up) sensory cortical processing of audiovisual speech. When compared to the NH listeners, the CI candidates (before implantation) and the CI users (after implantation) exhibited enhanced lipreading abilities and an altered cortical response at the N1 latency range (90-150 ms) that was characterized by a decreased theta oscillation power (4-8 Hz) and a smaller amplitude in the auditory cortex. After implantation, however, the auditory-cortex response gradually increased and developed a stronger intra-modal connectivity. Nevertheless, task efficiency and activation in the visual cortex was significantly modulated in both groups by focusing attention on the visual as compared to the auditory speech signal, with the NH listeners additionally showing an attention-dependent decrease in beta oscillation power (13-30 Hz). In sum, these results suggest remarkable deprivation effects on audiovisual speech processing in the auditory cortex, which partially reverse after implantation. Although even experienced CI users still show distinct audiovisual speech processing compared to NH listeners, pronounced effects of (top-down) direction of attention on (bottom-up) audiovisual processing can be observed in both groups. However, NH listeners but not CI users appear to show enhanced allocation of cognitive resources in visually as compared to auditory attended audiovisual speech conditions, which supports our behavioural observations of poorer lipreading abilities and reduced visual influence on audition in NH listeners as compared to CI users.

Electrophysiological study of visual processing in children with cochlear implants

Electrophysiological studies of congenitally deaf children and adults have reported atypical visual evoked potentials (VEPs) which have been associated with both behavioral enhancements of visual attention as well as poorer performance and outcomes in tests of spoken language speech processing. This pattern has often been interpreted as a maladaptive consequence of early auditory deprivation, whereby a remapping of auditory cortex by the visual system ultimately reduces resources necessary for optimal rehabilitative outcomes of spoken language acquisition and use. Making use of a novel electrophysiological paradigm, we compare VEPs in children with severe to profound congenital deafness who received a cochlear implant(s) prior to 31 months (n = 28) and typically developing age matched controls (n = 28). We observe amplitude enhancements and in some cases latency differences in occipitally expressed P1 and N1 VEP components in CI-using children as well as an early frontal negativity, N1a. We relate these findings to developmental factors such as chronological age and spoken language understanding. We further evaluate whether VEPs are additionally modulated by auditory stimulation. Collectively, these data provide a means to examine the extent to which atypical VEPs are consistent with prior accounts of maladaptive cross-modal plasticity. Our results support a view that VEP changes reflect alterations to visual-sensory attention and saliency mechanisms rather than a re-mapping of auditory cortex. The present data suggests that early auditory deprivation may have temporally prolonged effects on visual system processing even after activation and use of cochlear implant.

Does age-related hearing loss deteriorate attentional resources?

Recent work suggests that age-related hearing loss (HL) is a possible risk factor for cognitive decline in older adults. Resulting poor speech recognition negatively impacts cognitive, social and emotional functioning and may relate to dementia. However, little is known about the consequences of hearing loss on other non-linguistic domains of cognition. The aim of this study was to investigate the role of HL on covert orienting of attention, selective attention and executive control. We compared older adults with and without mild to moderate hearing loss (26-60 dB) performing (1) a spatial cueing task with uninformative central cues (social vs. nonsocial cues), (2) a flanker task and (3) a neuropsychological assessment of attention. The results showed that overall response times and flanker interference effects were comparable across groups. However, in spatial cueing of attention using social and nonsocial cues, hearing impaired individuals were characterized by reduced validity effects, though no additional group differences were found between social and nonsocial cues. Hearing impaired individuals also demonstrated diminished performance on the Montreal Cognitive Assessment (MoCA) and on tasks requiring divided attention and flexibility. This work indicates that while response speed and response inhibition appear to be preserved following mild-to-moderate acquired hearing loss, orienting of attention, divided attention and the ability to flexibly allocate attentional resources are more deteriorated in older adults with HL. This work suggests that hearing loss might exacerbate the detrimental influences of aging on visual attention.

Dynamic networks differentiate the language ability of children with cochlear implants

Background

Cochlear implantation (CI) in prelingually deafened children has been shown to be an effective intervention for developing language and reading skill. However, there is a substantial proportion of the children receiving CI who struggle with language and reading. The current study-one of the first to implement electrical source imaging in CI population was designed to identify the neural underpinnings in two groups of CI children with good and poor language and reading skill.

Methods

Data using high density electroencephalography (EEG) under a resting state condition was obtained from 75 children, 50 with CIs having good (HL) or poor language skills (LL) and 25 normal hearing (NH) children. We identified coherent sources using dynamic imaging of coherent sources (DICS) and their effective connectivity computing time-frequency causality estimation based on temporal partial directed coherence (TPDC) in the two CI groups compared to a cohort of age and gender matched NH children.

Findings

Sources with higher coherence amplitude were observed in three frequency bands (alpha, beta and gamma) for the CI groups when compared to normal hearing children. The two groups of CI children with good (HL) and poor (LL) language ability exhibited not only different cortical and subcortical source profiles but also distinct effective connectivity between them. Additionally, a support vector machine (SVM) algorithm using these sources and their connectivity patterns for each CI group across the three frequency bands was able to predict the language and reading scores with high accuracy.

Interpretation

Increased coherence in the CI groups suggest overall that the oscillatory activity in some brain areas become more strongly coupled compared to the NH group. Moreover, the different sources and their connectivity patterns and their association to language and reading skill in both groups, suggest a compensatory adaptation that either facilitated or impeded language and reading development. The neural differences in the two groups of CI children may reflect potential biomarkers for predicting outcome success in CI children.

Crossmodal plasticity in hearing loss

Crossmodal plasticity is a textbook example of the ability of the brain to reorganize based on use. We review evidence from the auditory system showing that such reorganization has significant limits, is dependent on pre-existing circuitry and top-down interactions, and that extensive reorganization is often absent. We argue that the evidence does not support the hypothesis that crossmodal reorganization is responsible for closing critical periods in deafness, and crossmodal plasticity instead represents a neuronal process that is dynamically adaptable. We evaluate the evidence for crossmodal changes in both developmental and adult-onset deafness, which start as early as mild-moderate hearing loss and show reversibility when hearing is restored. Finally, crossmodal plasticity does not appear to affect the neuronal preconditions for successful hearing restoration. Given its dynamic and versatile nature, we describe how this plasticity can be exploited for improving clinical outcomes after neurosensory restoration.

Evidence of visual crossmodal reorganization positively relates to speech outcomes in cochlear implant users

Deaf individuals who use a cochlear implant (CI) have remarkably different outcomes for auditory speech communication ability. One factor assumed to affect CI outcomes is visual crossmodal plasticity in auditory cortex, where deprived auditory regions begin to support non-auditory functions such as vision. Previous research has viewed crossmodal plasticity as harmful for speech outcomes for CI users if it interferes with sound processing, while others have demonstrated that plasticity related to visual language may be beneficial for speech recovery. To clarify, we used electroencephalography (EEG) to measure brain responses to a partial face speaking a silent single-syllable word (visual language) in 15 CI users and 13 age-matched typical-hearing controls. We used source analysis on EEG activity to measure crossmodal visual responses in auditory cortex and then compared them to CI users' speech-in-noise listening ability. CI users' brain response to the onset of the video stimulus (face) was larger than controls in left auditory cortex, consistent with crossmodal activation after deafness. CI users also produced a mixture of alpha (8-12 Hz) synchronization and desynchronization in auditory cortex while watching lip movement while controls instead showed desynchronization. CI users with higher speech scores had stronger crossmodal responses in auditory cortex to the onset of the video, but those with lower speech scores had increases in alpha power during lip movement in auditory areas. Therefore, evidence of crossmodal reorganization in CI users does not necessarily predict poor speech outcomes, and differences in crossmodal activation during lip reading may instead relate to strategies or differences that CI users use in audiovisual speech communication.

Gamma-Band Modulation in Parietal Area as the Electroencephalographic Signature for Performance in Auditory-Verbal Working Memory: An Exploratory Pilot Study in Hearing and Unilateral Cochlear Implant Children

This pilot study investigates the neurophysiological patterns of visual and auditory verbal working memory (VWM) in unilateral cochlear implant users (UCIs). We compared the task-related electroencephalogram (EEG) power spectral density of 7- to 13-year-old UCIs (n = 7) with a hearing control group (HC, n = 10) during the execution of a three-level n-back task with auditory and visual verbal (letters) stimuli. Performances improved as memory load decreased regardless of sensory modality (SM) and group factors. Theta EEG activation over the frontal area was proportionally influenced by task level; the left hemisphere (LH) showed greater activation in the gamma band, suggesting lateralization of VWM function regardless of SM. However, HCs showed stronger activation patterns in the LH than UCIs regardless of SM and in the parietal area (PA) during the most challenging audio condition. Linear regressions for gamma activation in the PA suggest the presence of a pattern-supporting auditory VWM only in HCs. Our findings seem to recognize gamma activation in the PA as the signature of effective auditory VWM. These results, although preliminary, highlight this EEG pattern as a possible cause of the variability found in VWM outcomes in deaf children, opening up new possibilities for interdisciplinary research and rehabilitation intervention.

The timecourse of multisensory speech processing in unilaterally stimulated cochlear implant users revealed by ERPs

•

Both normal-hearing (NH) and cochlear implant (CI) users show a clear benefit in multisensory speech processing.

•

Group differences in ERP topographies and cortical source activation suggest distinct audiovisual speech processing in CI users when compared to NH listeners.

•

Electrical neuroimaging, including topographic and ERP source analysis, provides a suitable tool to study the timecourse of multisensory speech processing in CI users.

A cochlear implant (CI) is an auditory prosthesis which can partially restore the auditory function in patients with severe to profound hearing loss. However, this bionic device provides only limited auditory information, and CI patients may compensate for this limitation by means of a stronger interaction between the auditory and visual system. To better understand the electrophysiological correlates of audiovisual speech perception, the present study used electroencephalography (EEG) and a redundant target paradigm. Postlingually deafened CI users and normal-hearing (NH) listeners were compared in auditory, visual and audiovisual speech conditions. The behavioural results revealed multisensory integration for both groups, as indicated by shortened response times for the audiovisual as compared to the two unisensory conditions. The analysis of the N1 and P2 event-related potentials (ERPs), including topographic and source analyses, confirmed a multisensory effect for both groups and showed a cortical auditory response which was modulated by the simultaneous processing of the visual stimulus. Nevertheless, the CI users in particular revealed a distinct pattern of N1 topography, pointing to a strong visual impact on auditory speech processing. Apart from these condition effects, the results revealed ERP differences between CI users and NH listeners, not only in N1/P2 ERP topographies, but also in the cortical source configuration. When compared to the NH listeners, the CI users showed an additional activation in the visual cortex at N1 latency, which was positively correlated with CI experience, and a delayed auditory-cortex activation with a reversed, rightward functional lateralisation. In sum, our behavioural and ERP findings demonstrate a clear audiovisual benefit for both groups, and a CI-specific alteration in cortical activation at N1 latency when auditory and visual input is combined. These cortical alterations may reflect a compensatory strategy to overcome the limited CI input, which allows the CI users to improve the lip-reading skills and to approximate the behavioural performance of NH listeners in audiovisual speech conditions. Our results are clinically relevant, as they highlight the importance of assessing the CI outcome not only in auditory-only, but also in audiovisual speech conditions.