Fundamental frequency is critical to speech perception in noise in combined acoustic and electric hearing

Comparison of Tonotopic and Default Frequency Fitting for Speech Understanding in Noise in New Cochlear Implantees: A Prospective, Randomized, Double-Blind, Cross-Over Study

OBJECTIVES

While cochlear implants (CIs) have provided benefits for speech recognition in quiet for subjects with severe-to-profound hearing loss, speech recognition in noise remains challenging. A body of evidence suggests that reducing frequency-to-place mismatch may positively affect speech perception. Thus, a fitting method based on a tonotopic map may improve speech perception results in quiet and noise. The aim of our study was to assess the impact of a tonotopic map on speech perception in noise and quiet in new CI users.

DESIGN

A prospective, randomized, double-blind, two-period cross-over study in 26 new CI users was performed over a 6-month period. New CI users older than 18 years with bilateral severe-to-profound sensorineural hearing loss or complete hearing loss for less than 5 years were selected in the University Hospital Centre of Rennes in France. An anatomical tonotopic map was created using postoperative flat-panel computed tomography and a reconstruction software based on the Greenwood function. Each participant was randomized to receive a conventional map followed by a tonotopic map or vice versa. Each setting was maintained for 6 weeks, at the end of which participants performed speech perception tasks. The primary outcome measure was speech recognition in noise. Participants were allocated to sequences by block randomization of size two with a ratio 1:1 (CONSORT Guidelines). Participants and those assessing the outcomes were blinded to the intervention.

RESULTS

Thirteen participants were randomized to each sequence. Two of the 26 participants recruited (one in each sequence) had to be excluded due to the COVID-19 pandemic. Twenty-four participants were analyzed. Speech recognition in noise was significantly better with the tonotopic fitting at all signal-to-noise ratio (SNR) levels tested [SNR = +9 dB, p = 0.002, mean effect (ME) = 12.1%, 95% confidence interval (95% CI) = 4.9 to 19.2, standardized effect size (SES) = 0.71; SNR = +6 dB, p < 0.001, ME = 16.3%, 95% CI = 9.8 to 22.7, SES = 1.07; SNR = +3 dB, p < 0.001 ME = 13.8%, 95% CI = 6.9 to 20.6, SES = 0.84; SNR = 0 dB, p = 0.003, ME = 10.8%, 95% CI = 4.1 to 17.6, SES = 0.68]. Neither period nor interaction effects were observed for any signal level. Speech recognition in quiet ( p = 0.66) and tonal audiometry ( p = 0.203) did not significantly differ between the two settings. 92% of the participants kept the tonotopy-based map after the study period. No correlation was found between speech-in-noise perception and age, duration of hearing deprivation, angular insertion depth, or position or width of the frequency filters allocated to the electrodes.

CONCLUSION

For new CI users, tonotopic fitting appears to be more efficient than the default frequency fitting because it allows for better speech recognition in noise without compromising understanding in quiet.

The effects of speech masking on neural tracking of acoustic and semantic features of natural speech

Listening environments contain background sounds that mask speech and lead to communication challenges. Sensitivity to slow acoustic fluctuations in speech can help segregate speech from background noise. Semantic context can also facilitate speech perception in noise, for example, by enabling prediction of upcoming words. However, not much is known about how different degrees of background masking affect the neural processing of acoustic and semantic features during naturalistic speech listening. In the current electroencephalography (EEG) study, participants listened to engaging, spoken stories masked at different levels of multi-talker babble to investigate how neural activity in response to acoustic and semantic features changes with acoustic challenges, and how such effects relate to speech intelligibility. The pattern of neural response amplitudes associated with both acoustic and semantic speech features across masking levels was U-shaped, such that amplitudes were largest for moderate masking levels. This U-shape may be due to increased attentional focus when speech comprehension is challenging, but manageable. The latency of the neural responses increased linearly with increasing background masking, and neural latency change associated with acoustic processing most closely mirrored the changes in speech intelligibility. Finally, tracking responses related to semantic dissimilarity remained robust until severe speech masking (-3 dB SNR). The current study reveals that neural responses to acoustic features are highly sensitive to background masking and decreasing speech intelligibility, whereas neural responses to semantic features are relatively robust, suggesting that individuals track the meaning of the story well even in moderate background sound.

Considerations in pediatric device candidacy: An emphasis on spoken language

As cochlear implant (CI) candidacy expands to consider children with more residual hearing, the use of a CI and a hearing aid (HA) at the non-implanted ear (bimodal devices) is increasing. This case study examines the contributions of acoustic and electric input to speech perception performance for a pediatric bimodal device user (S1) who is a borderline bilateral cochlear implant candidate. S1 completed a battery of perceptual tests in CI-only, HA-only and bimodal conditions. Since CIs and HAs differ in their ability to transmit cues related to segmental and suprasegmental perception, both types of perception were tested. Performance in all three device conditions were generally similar across tests, showing no clear device-condition benefit. Further, S1's spoken language performance was compared to those of a large group of children with prelingual severe-profound hearing loss who used two devices from a young age, at least one of which was a CI. S1's speech perception and language scores were average or above-average compared to these other pediatric CI recipients. Both segmental and suprasegmental speech perception, and spoken language skills should be examined to determine the broad-scale performance level of bimodal recipients, especially when deciding whether to move from bimodal devices to bilateral CIs.

Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding

Understanding speech in background noise is challenging. Wearing face-masks, imposed by the COVID19-pandemics, makes it even harder. We developed a multi-sensory setup, including a sensory substitution device (SSD) that can deliver speech simultaneously through audition and as vibrations on the fingertips. The vibrations correspond to low frequencies extracted from the speech input. We trained two groups of non-native English speakers in understanding distorted speech in noise. After a short session (30-45 min) of repeating sentences, with or without concurrent matching vibrations, we showed comparable mean group improvement of 14-16 dB in Speech Reception Threshold (SRT) in two test conditions, i.e., when the participants were asked to repeat sentences only from hearing and also when matching vibrations on fingertips were present. This is a very strong effect, if one considers that a 10 dB difference corresponds to doubling of the perceived loudness. The number of sentence repetitions needed for both types of training to complete the task was comparable. Meanwhile, the mean group SNR for the audio-tactile training (14.7 ± 8.7) was significantly lower (harder) than for the auditory training (23.9 ± 11.8), which indicates a potential facilitating effect of the added vibrations. In addition, both before and after training most of the participants (70-80%) showed better performance (by mean 4-6 dB) in speech-in-noise understanding when the audio sentences were accompanied with matching vibrations. This is the same magnitude of multisensory benefit that we reported, with no training at all, in our previous study using the same experimental procedures. After training, performance in this test condition was also best in both groups (SRT ~ 2 dB). The least significant effect of both training types was found in the third test condition, i.e. when participants were repeating sentences accompanied with non-matching tactile vibrations and the performance in this condition was also poorest after training. The results indicate that both types of training may remove some level of difficulty in sound perception, which might enable a more proper use of speech inputs delivered via vibrotactile stimulation. We discuss the implications of these novel findings with respect to basic science. In particular, we show that even in adulthood, i.e. long after the classical "critical periods" of development have passed, a new pairing between a certain computation (here, speech processing) and an atypical sensory modality (here, touch) can be established and trained, and that this process can be rapid and intuitive. We further present possible applications of our training program and the SSD for auditory rehabilitation in patients with hearing (and sight) deficits, as well as healthy individuals in suboptimal acoustic situations.

Is Characteristic Frequency Limiting Real-Time Electrocochleography During Cochlear Implantation?

Objectives

Electrocochleography (ECochG) recordings during cochlear implantation have shown promise in estimating the impact on residual hearing. The purpose of the study was (1) to determine whether a 250-Hz stimulus is superior to 500-Hz in detecting residual hearing decrement and if so; (2) to evaluate whether crossing the 500-Hz tonotopic, characteristic frequency (CF) place partly explains the problems experienced using 500-Hz.

Design

Multifrequency ECochG comprising an alternating, interleaved acoustic complex of 250- and 500-Hz stimuli was used to elicit cochlear microphonics (CMs) during insertion. The largest ECochG drops (≥30% reduction in CM) were identified. After insertion, ECochG responses were measured using the individual electrodes along the array for both 250- and 500-Hz stimuli. Univariate regression was used to predict whether 250- or 500-Hz CM drops explained low-frequency pure tone average (LFPTA; 125-, 250-, and 500-Hz) shift at 1-month post-activation. Postoperative CT scans were performed to evaluate cochlear size and angular insertion depth.

Results

For perimodiolar insertions (N = 34), there was a stronger linear correlation between the largest ECochG drop using 250-Hz stimulus and LFPTA shift (r = 0.58), compared to 500-Hz (r = 0.31). The 250- and 500-Hz CM insertion tracings showed an amplitude peak at two different locations, with the 500-Hz peak occurring earlier in most cases than the 250-Hz peak, consistent with tonotopicity. When using the entire array for recordings after insertion, a maximum 500-Hz response was observed 2-6 electrodes basal to the most-apical electrode in 20 cases (58.9%). For insertions where the apical insertion angle is >350 degrees and the cochlear diameter is <9.5 mm, the maximum 500-Hz ECochG response may occur at the non-apical most electrode. For lateral wall insertions (N = 14), the maximum 250- and 500-Hz CM response occurred at the most-apical electrode in all but one case.

Conclusion

Using 250-Hz stimulus for ECochG feedback during implantation is more predictive of hearing preservation than 500-Hz. This is due to the electrode passing the 500-Hz CF during insertion which may be misidentified as intracochlear trauma; this is particularly important in subjects with smaller cochlear diameters and deeper insertions. Multifrequency ECochG can be used to differentiate between trauma and advancement of the apical electrode beyond the CF.

Relationship Between Intraoperative Electrocochleography and Hearing Preservation

OBJECTIVES

To compare intraoperative intracochlear electrocochleography (ECochG) with hearing preservation outcomes in cochlear implant (CI) subjects.

DESIGN

Intraoperative electrocochleography was performed in adult CI subjects who were recipients of Advanced Bionics' Bionics LLC precurved HiFocus MidScala or straight HiFocus SlimJ electrode arrays. ECochG responses were recorded from the most apical electrode contact during insertion. No changes to the insertions were made due to ECochG monitoring. No information about insertion resistance was collected. ECochG drops were estimated as the change in amplitude from peak (defined as maximum amplitude response) to drop (largest drop) point after the peak during insertion was measured following the peak response. Audiometric thresholds from each subject were obtained before and approximately 1 month after CI surgery. The change in pure tone average for frequencies between 125 Hz and 500 Hz was measured after surgery. No postoperative CT scans were collected as part of this study.

RESULTS

A total of 68 subjects from five surgical centers participated in the study. The study sample included 30 MidScala and 38 SlimJ electrodes implanted by approximately 20 surgeons who contributed to the study. Although a wide range of results were observed, there was a moderate positive correlation (Pearson Correlation coefficient, r = 0.56, p < 0.01) between the size of the ECochG drop and the magnitude of pure tone average change. This trend was present for both the MidScala and SlimJ arrays. The SlimJ and MidScala arrays produced significantly different hearing loss after surgery.

CONCLUSION

Large ECochG amplitude drops observed during electrode insertion indicated poorer hearing preservation. Although the outcomes were variable, this information may be helpful to guide surgical decision-making when contemplating full electrode insertion and the likelihood of hearing preservation.

Phantom Stimulation for Cochlear Implant Users With Residual Low-Frequency Hearing

In cochlear implants (CIs), phantom stimulation can be used to extend the pitch range toward apical regions of the cochlea. Phantom stimulation consists of partial bipolar stimulation, in which current is distributed across two intracochlear electrodes and one extracochlear electrode as defined by the compensation coefficient σ. The aim of this study was, (1) to evaluate the benefit of conveying low-frequency information through phantom stimulation for cochlear implant (CI) subjects with low-frequency residual hearing using electric stimulation alone, (2) to compare the speech reception thresholds obtained from electric-acoustic stimulation (EAS) and electric stimulation in combination with phantom stimulation (EPS), and (3) to investigate the effect of spectrally overlapped bandwidth of speech conveyed via simultaneous acoustic and phantom stimulation on speech reception thresholds.

Electro-Tactile Stimulation Enhances Cochlear-Implant Melody Recognition: Effects of Rhythm and Musical Training

OBJECTIVES

Electro-acoustic stimulation (EAS) enhances speech and music perception in cochlear-implant (CI) users who have residual low-frequency acoustic hearing. For CI users who do not have low-frequency acoustic hearing, tactile stimulation may be used in a similar fashion as residual low-frequency acoustic hearing to enhance CI performance. Previous studies showed that electro-tactile stimulation (ETS) enhanced speech recognition in noise and tonal language perception for CI listeners. Here, we examined the effect of ETS on melody recognition in both musician and nonmusician CI users.

DESIGN

Nine musician and eight nonmusician CI users were tested in a melody recognition task with or without rhythmic cues in three testing conditions: CI only (E), tactile only (T), and combined CI and tactile stimulation (ETS).

RESULTS

Overall, the combined electrical and tactile stimulation enhanced the melody recognition performance in CI users by 9% points. Two additional findings were observed. First, musician CI users outperformed nonmusicians CI users in melody recognition, but the size of the enhancement effect was similar between the two groups. Second, the ETS enhancement was significantly higher with nonrhythmic melodies than rhythmic melodies in both groups.

CONCLUSIONS

These findings suggest that, independent of musical experience, the size of the ETS enhancement depends on integration efficiency between tactile and auditory stimulation, and that the mechanism of the ETS enhancement is improved electric pitch perception. The present study supports the hypothesis that tactile stimulation can be used to improve pitch perception in CI users.

Speech Segregation in Active Middle Ear Stimulation: Masking Release With Changing Fundamental Frequency

OBJECTIVES

Temporal fine structure information such as low-frequency sounds including the fundamental frequency (F0) is important to separate different talkers in noisy environments. Speech perception in noise is negatively affected by reduced temporal fine structure resolution in cochlear hearing loss. It has been shown that normal-hearing (NH) people as well as cochlear implant patients with preserved acoustic low-frequency hearing benefit from different F0 between concurrent talkers. Though patients with an active middle ear implant (AMEI) report better sound quality compared with hearing aids, they often struggle when listening in noise. The primary objective was to evaluate whether or not patients with a Vibrant Soundbridge AMEI were able to benefit from F0 differences in a concurrent talker situation and if the effect was comparable to NH individuals.

DESIGN

A total of 13 AMEI listeners and 13 NH individuals were included. A modified variant of the Oldenburg sentence test was used to emulate a concurrent talker scenario. One sentence from the test corpus served as the masker and the remaining sentences as target speech. The F0 of the masker sentence was shifted upward by 4, 8, and 12 semitones. The target and masker sentences were presented simultaneously to the study subjects and the speech reception threshold was assessed by adaptively varying the masker level. To evaluate any impact of the occlusion effect on speech perception, AMEI listeners were tested in two configurations: with a plugged ear-canal contralateral to the implant side, indicated as AMEIcontra, or with both ears plugged, indicated as AMEIboth.

RESULTS

In both study groups, speech perception improved when the F0 difference between target and masker increased. This was significant when the difference was at least 8 semitones; the F0-based release from masking was 3.0 dB in AMEIcontra (p = 0.009) and 2.9 dB in AMEIboth (p = 0.015), compared with 5.6 dB in NH listeners (p < 0.001). A difference of 12 semitones revealed a F0-based release from masking of 3.5 dB in the AMEIcontra (p = 0.002) and 3.4 dB in the AMEIboth (p = 0.003) condition, compared with 5.0 dB in NH individuals (p < 0.001).

CONCLUSIONS

Though AMEI users deal with problems resulting from cochlear damage, hearing amplification with the implant enables a masking release based on F0 differences when F0 between a target and masker sentence was at least 8 semitones. Additional occlusion of the ear canal on the implant side did not affect speech performance. The current results complement the knowledge about the benefit of F0 within the acoustic low-frequency hearing.

The Temporal Fine Structure of Background Noise Determines the Benefit of Bimodal Hearing for Recognizing Speech

Cochlear implant (CI) users have more difficulty understanding speech in temporally modulated noise than in steady-state (SS) noise. This is thought to be caused by the limited low-frequency information that CIs provide, as well as by the envelope coding in CIs that discards the temporal fine structure (TFS). Contralateral amplification with a hearing aid, referred to as bimodal hearing, can potentially provide CI users with TFS cues to complement the envelope cues provided by the CI signal. In this study, we investigated whether the use of a CI alone provides access to only envelope cues and whether acoustic amplification can provide additional access to TFS cues. To this end, we evaluated speech recognition in bimodal listeners, using SS noise and two amplitude-modulated noise types, namely babble noise and amplitude-modulated steady-state (AMSS) noise. We hypothesized that speech recognition in noise depends on the envelope of the noise, but not on its TFS when listening with a CI. Secondly, we hypothesized that the amount of benefit gained by the addition of a contralateral hearing aid depends on both the envelope and TFS of the noise. The two amplitude-modulated noise types decreased speech recognition more effectively than SS noise. Against expectations, however, we found that babble noise decreased speech recognition more effectively than AMSS noise in the CI-only condition. Therefore, we rejected our hypothesis that TFS is not available to CI users. In line with expectations, we found that the bimodal benefit was highest in babble noise. However, there was no significant difference between the bimodal benefit obtained in SS and AMSS noise. Our results suggest that a CI alone can provide TFS cues and that bimodal benefits in noise depend on TFS, but not on the envelope of the noise.

Evaluation of a novel bimodal fitting formula in Advanced Bionics cochlear implant recipients

Purpose: The study's objectives were to (1) evaluate benefit from a novel bimodal fitting formula (Adaptive Phonak Digital Bimodal Fitting Formula [APDB]), and (2) compare outcomes with APDB and a traditional fitting formula (NAL-NL2). Methods: This prospective study evaluated outcomes in ten adults with unilateral Advanced Bionics (AB) cochlear implants (CI). Participants were tested bimodally with NAL-NL2 and APDB programed on Naída Link UP HAs. Measures of speech perception, sound quality, and preference were obtained with two bimodal configurations (CI + HANAL-NL2 and CI + HAAPDB). Participants used the CI + HAAPDB configuration for an acclimation period, after which measures were repeated. Results: Significant bimodal benefit was measured from both HA fitting formulae for speech perception in noise compared to the CI-only condition. Improved individual outcomes with the APDB were observed, but group differences were not statistically significant. Participants reported subjective benefit from APDB on blind comparisons of preference and sound quality. Conclusions: Significant benefit was found with both bimodal conditions compared to the CI-only condition; however, bimodal speech perception results were not significantly different. Users reported benefit from the APDB formula over NAL-NL2 formula. Due to individual improved speech perception and overall subjective preference for APDB, clinicians should consider APDB with AB CI recipients.

Brief Report: Speech-in-Noise Recognition and the Relation to Vocal Pitch Perception in Adults with Autism Spectrum Disorder and Typical Development

We tested the ability to recognise speech-in-noise and its relation to the ability to discriminate vocal pitch in adults with high-functioning autism spectrum disorder (ASD) and typically developed adults (matched pairwise on age, sex, and IQ). Typically developed individuals understood speech in higher noise levels as compared to the ASD group. Within the control group but not within the ASD group, better speech-in-noise recognition abilities were significantly correlated with better vocal pitch discrimination abilities. Our results show that speech-in-noise recognition is restricted in people with ASD. We speculate that perceptual impairments such as difficulties in vocal pitch perception might be relevant in explaining these difficulties in ASD.

Perceptual contributions of vowels and consonant-vowel transitions in simulated electric-acoustic hearing

The present work assessed the perceptual impact of vowels and consonant-vowel transitions in simulated electric-acoustic hearing. Mandarin sentences were processed by two vocoder models simulating electric and electric-acoustic stimulation, followed by a noise-replacement paradigm to preserve target speech segments and replace the rest with noise, and finally presented to normal-hearing listeners to recognize. The results demonstrated a much larger perceptual advantage of vowels than consonants to sentence intelligibility, and showed the combined-stimulation advantage under segmental conditions containing vowels. Adding consonant-vowel transitions in combined electric-acoustic stimulation yielded sentence recognition performance equivalent to that observed with electric stimulation and full speech segments.

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

BACKGROUND

Hearing loss is becoming a real social and health problem. Its prevalence in the elderly is an epidemic. The risk of developing hearing loss is also growing among younger people. If left untreated, hearing loss can perpetuate development of neurodegenerative diseases, including dementia. Despite recent advancements in hearing aid (HA) and cochlear implant (CI) technologies, hearing impaired users still encounter significant practical and social challenges, with or without aids. In particular, they all struggle with understanding speech in challenging acoustic environments, especially in presence of a competing speaker.

OBJECTIVES

In the current proof-of-concept study we tested whether multisensory stimulation, pairing audition and a minimal-size touch device would improve intelligibility of speech in noise.

METHODS

To this aim we developed an audio-to-tactile sensory substitution device (SSD) transforming low-frequency speech signals into tactile vibrations delivered on two finger tips. Based on the inverse effectiveness law, i.e., multisensory enhancement is strongest when signal-to-noise ratio is lowest between senses, we embedded non-native language stimuli in speech-like noise and paired it with a low-frequency input conveyed through touch.

RESULTS

We found immediate and robust improvement in speech recognition (i.e. in the Signal-To-Noise-ratio) in the multisensory condition without any training, at a group level as well as in every participant. The reported improvement at the group-level of 6 dB was indeed major considering that an increase of 10 dB represents a doubling of the perceived loudness.

CONCLUSIONS

These results are especially relevant when compared to previous SSD studies showing effects in behavior only after a demanding cognitive training. We discuss the implications of our results for development of SSDs and of specific rehabilitation programs for the hearing impaired either using or not using HAs or CIs. We also discuss the potential application of such a set-up for sense augmentation, such as when learning a new language.

Effect of Dual-Carrier Processing on the Intelligibility of Concurrent Vocoded Sentences

Purpose

The goal of this study was to examine the role of carrier cues in sound source segregation and the possibility to enhance the intelligibility of 2 sentences presented simultaneously. Dual-carrier (DC) processing (Apoux, Youngdahl, Yoho, & Healy, 2015) was used to introduce synthetic carrier cues in vocoded speech.

Method

Listeners with normal hearing heard sentences processed either with a DC or with a traditional single-carrier (SC) vocoder. One group was asked to repeat both sentences in a sentence pair (Experiment 1). The other group was asked to repeat only 1 sentence of the pair and was provided additional segregation cues involving onset asynchrony (Experiment 2).

Results

Both experiments showed that not only is the "target" sentence more intelligible in DC compared with SC, but the "background" sentence intelligibility is equally enhanced. The participants did not benefit from the additional segregation cues.

Conclusions

The data showed a clear benefit of using a distinct carrier to convey each sentence (i.e., DC processing). Accordingly, the poor speech intelligibility in noise typically observed with SC-vocoded speech may be partly attributed to the envelope of independent sound sources sharing the same carrier. Moreover, this work suggests that noise reduction may not be the only viable option to improve speech intelligibility in noise for users of cochlear implants. Alternative approaches aimed at enhancing sound source segregation such as DC processing may help to improve speech intelligibility while preserving and enhancing the background.

Effects of fundamental frequency contour on understanding Mandarin sentences in bimodal hearing simulations

Fundamental frequency (F0) contour carries important information for understanding a tonal language. The present work assessed the effects of F0 contour on understanding Mandarin sentences in bimodal hearing simulations, including three conditions of acoustic-only, electric-only, and combined stimulations. Test stimuli were synthesized Mandarin sentences, each word with a normal, flat, or randomly assigned lexical tone, and presented to normal-hearing Mandarin-speaking listeners to recognize. Experimental results showed that changing F0 contour significantly affected the perception of Mandarin sentences under all conditions of acoustic-only, electric-only, and combined stimulations. The combined-stimulation advantage was only observed for test stimuli with the normal F0 contour.

Relationship between speech-evoked neural responses and perception of speech in noise in older adults

Speech-in-noise (SPIN) perception involves neural encoding of temporal acoustic cues. Cues include temporal fine structure (TFS) and envelopes that modulate at syllable (Slow-rate ENV) and fundamental frequency (F₀-rate ENV) rates. Here the relationship between speech-evoked neural responses to these cues and SPIN perception was investigated in older adults. Theta-band phase-locking values (PLVs) that reflect cortical sensitivity to Slow-rate ENV and peripheral/brainstem frequency-following responses phase-locked to F₀-rate ENV (FFR_{ENV_F0}) and TFS (FFR_TFS) were measured from scalp-electroencephalography responses to a repeated speech syllable in steady-state speech-shaped noise (SpN) and 16-speaker babble noise (BbN). The results showed that (1) SPIN performance and PLVs were significantly higher under SpN than BbN, implying differential cortical encoding may serve as the neural mechanism of SPIN performance that varies as a function of noise types; (2) PLVs and FFR_TFS at resolved harmonics were significantly related to good SPIN performance, supporting the importance of phase-locked neural encoding of Slow-rate ENV and TFS of resolved harmonics during SPIN perception; (3) FFR_{ENV_F0} was not associated to SPIN performance until audiometric threshold was controlled for, indicating that hearing loss should be carefully controlled when studying the role of neural encoding of F₀-rate ENV. Implications are drawn with respect to fitting auditory prostheses.

Top-Down Processes in Simulated Electric-Acoustic Hearing: The Effect of Linguistic Context on Bimodal Benefit for Temporally Interrupted Speech

OBJECTIVES

Previous studies have documented the benefits of bimodal hearing as compared with a cochlear implant alone, but most have focused on the importance of bottom-up, low-frequency cues. The purpose of the present study was to evaluate the role of top-down processing in bimodal hearing by measuring the effect of sentence context on bimodal benefit for temporally interrupted sentences. It was hypothesized that low-frequency acoustic cues would facilitate the use of contextual information in the interrupted sentences, resulting in greater bimodal benefit for the higher context (CUNY) sentences than for the lower context (IEEE) sentences.

DESIGN

Young normal-hearing listeners were tested in simulated bimodal listening conditions in which noise band vocoded sentences were presented to one ear with or without low-pass (LP) filtered speech or LP harmonic complexes (LPHCs) presented to the contralateral ear. Speech recognition scores were measured in three listening conditions: vocoder-alone, vocoder combined with LP speech, and vocoder combined with LPHCs. Temporally interrupted versions of the CUNY and IEEE sentences were used to assess listeners' ability to fill in missing segments of speech by using top-down linguistic processing. Sentences were square-wave gated at a rate of 5 Hz with a 50% duty cycle. Three vocoder channel conditions were tested for each type of sentence (8, 12, and 16 channels for CUNY; 12, 16, and 32 channels for IEEE) and bimodal benefit was compared for similar amounts of spectral degradation (matched-channel comparisons) and similar ranges of baseline performance. Two gain measures, percentage-point gain and normalized gain, were examined.

RESULTS

Significant effects of context on bimodal benefit were observed when LP speech was presented to the residual-hearing ear. For the matched-channel comparisons, CUNY sentences showed significantly higher normalized gains than IEEE sentences for both the 12-channel (20 points higher) and 16-channel (18 points higher) conditions. For the individual gain comparisons that used a similar range of baseline performance, CUNY sentences showed bimodal benefits that were significantly higher (7% points, or 15 points normalized gain) than those for IEEE sentences. The bimodal benefits observed here for temporally interrupted speech were considerably smaller than those observed in an earlier study that used continuous speech. Furthermore, unlike previous findings for continuous speech, no bimodal benefit was observed when LPHCs were presented to the LP ear.

CONCLUSIONS

Findings indicate that linguistic context has a significant influence on bimodal benefit for temporally interrupted speech and support the hypothesis that low-frequency acoustic information presented to the residual-hearing ear facilitates the use of top-down linguistic processing in bimodal hearing. However, bimodal benefit is reduced for temporally interrupted speech as compared with continuous speech, suggesting that listeners' ability to restore missing speech information depends not only on top-down linguistic knowledge but also on the quality of the bottom-up sensory input.

Electro-tactile stimulation (ETS) enhances cochlear-implant Mandarin tone recognition

Objective

Electro-acoustic stimulation (EAS) is an effective method to enhance cochlear-implant performance in individuals who have residual low-frequency acoustic hearing. To help the majority of cochlear implant users who do not have any functional residual acoustic hearing, electro-tactile stimulation (ETS) may be used because tactile sensation has a frequency range and perceptual capabilities similar to that produced by acoustic stimulation in the EAS users.

Methods

Following up the first ETS study showing enhanced English sentence recognition in noise,¹ the present study evaluated the effect of ETS on Mandarin tone recognition in noise in two groups of adult Mandarin-speaking individuals. The first group included 11 normal-hearing individuals who listened to a 4-channel, noise-vocoded, cochlear-implant simulation. The second group included 1 unilateral cochlear-implant user and 2 bilateral users with each of their devices being tested independently. Both groups participated in a 4-alternative, forced-choice task, in which they had to identify a tone that was presented in noise at a 0-dB signal-to-noise ratio via electric stimulation (actual or simulated cochlear implants), tactile stimulation or the combined ETS.

Results

While electric or tactile stimulation alone produced similar tone recognition (∼40% correct), the ETS enhanced the cochlear-implant tone recognition by 17–18 percentage points. The size of the present ETS enhancement effect was similar to that of the previously reported EAS effect on Mandarin tone recognition. Psychophysical analysis on tactile sensation showed an important role of frequency discrimination in the ETS enhancement.

Conclusion

Tactile stimulation can potentially enhance Mandarin tone recognition in cochlear-implant users who do not have usable residual acoustic hearing. To optimize this potential, high fundamental frequencies need to be transposed to a 100–200 Hz range.

Vibro-Tactile Enhancement of Speech Intelligibility in Multi-talker Noise for Simulated Cochlear Implant Listening

Many cochlear implant (CI) users achieve excellent speech understanding in acoustically quiet conditions but most perform poorly in the presence of background noise. An important contributor to this poor speech-in-noise performance is the limited transmission of low-frequency sound information through CIs. Recent work has suggested that tactile presentation of this low-frequency sound information could be used to improve speech-in-noise performance for CI users. Building on this work, we investigated whether vibro-tactile stimulation can improve speech intelligibility in multi-talker noise. The signal used for tactile stimulation was derived from the speech-in-noise using a computationally inexpensive algorithm. Eight normal-hearing participants listened to CI simulated speech-in-noise both with and without concurrent tactile stimulation of their fingertip. Participants' speech recognition performance was assessed before and after a training regime, which took place over 3 consecutive days and totaled around 30 min of exposure to CI-simulated speech-in-noise with concurrent tactile stimulation. Tactile stimulation was found to improve the intelligibility of speech in multi-talker noise, and this improvement was found to increase in size after training. Presentation of such tactile stimulation could be achieved by a compact, portable device and offer an inexpensive and noninvasive means for improving speech-in-noise performance in CI users.

Electro-Tactile Stimulation Enhances Cochlear Implant Speech Recognition in Noise

For cochlear implant users, combined electro-acoustic stimulation (EAS) significantly improves the performance. However, there are many more users who do not have any functional residual acoustic hearing at low frequencies. Because tactile sensation also operates in the same low frequencies (<500 Hz) as the acoustic hearing in EAS, we propose electro-tactile stimulation (ETS) to improve cochlear implant performance. In ten cochlear implant users, a tactile aid was applied to the index finger that converted voice fundamental frequency into tactile vibrations. Speech recognition in noise was compared for cochlear implants alone and for the bimodal ETS condition. On average, ETS improved speech reception thresholds by 2.2 dB over cochlear implants alone. Nine of the ten subjects showed a positive ETS effect ranging from 0.3 to 7.0 dB, which was similar to the amount of the previously-reported EAS benefit. The comparable results indicate similar neural mechanisms that underlie both the ETS and EAS effects. The positive results suggest that the complementary auditory and tactile modes also be used to enhance performance for normal hearing listeners and automatic speech recognition for machines.

Intracochlear Recordings of Acoustically and Electrically Evoked Potentials in Nucleus Hybrid L24 Cochlear Implant Users and Their Relationship to Speech Perception

The Hybrid cochlear implant (CI) has been developed for individuals with high frequency hearing loss who retain good low frequency hearing. Outcomes have been encouraging but individual variability is high; the health of the cochlea and the auditory nerve may be important factors driving outcomes. Electrically evoked compound action potentials (ECAPs) reflect the response of the auditory nerve to electrical stimulation while electrocochleography (ECochG) reflects the response of the cochlear hair cells and auditory nerve to acoustic stimulation. In this study both ECAPs and ECochG responses were recorded from Nucleus Hybrid L24 CI users. Correlations between these two measures of peripheral auditory function and speech perception are reported. This retrospective study includes data from 25 L24 CI users. ECAPs and ECochG responses were recorded from an intracochlear electrode using stimuli presented at or near maximum acceptable loudness levels. Speech perception was assessed using Consonant-Nucleus-Consonant (CNC) word lists presented in quiet and AzBio sentences presented at a +5 dB signal-to-noise ratio in both the combined acoustic and electric (A+E) and electric (E) alone listening modes. Acoustic gain was calculated by subtracting these two scores. Correlations between these physiologic and speech perception measures were then computed. ECAP amplitudes recorded from the most apical electrode were significantly correlated with CNC scores measured in the E alone (r = 0.56) and A+E conditions (r = 0.64), but not with performance on the AzBio test. ECochG responses recorded using the most apical electrode in the intracochlear array but evoked using a 500 Hz tone burst were not correlated with either the scores on the CNC or AzBio tests. However, ECochG amplitude was correlated with a composite metric relating the additional benefit of acoustic gain in noise relative to quiet conditions (r = 0.67). Both measures can be recorded from Hybrid L24 CI users and both ECAP and ECochG measures may result in more complete characterization of speech perception outcomes than either measure alone.

Robustness against distortion of fundamental frequency cues in simulated electro-acoustic hearing

Speech recognition by cochlear implant users can be improved by adding an audible low frequency acoustic signal to electrical hearing; the resulting improvement is deemed "electro-acoustic stimulation (EAS) benefit." However, a crucial low frequency cue, fundamental frequency (F0), can be distorted via the impaired auditory system. In order to understand how F0 distortions may affect EAS benefit, normal-hearing listeners were presented monaurally with vocoded speech (frequencies >250 Hz) and an acoustical signal (frequencies <250 Hz) with differing manipulations of the F0 signal, specifically: a pure tone with the correct mean F0 but with smaller variations around this mean, or a narrowband of white noise centered around F0, at varying bandwidths; a pure tone down-shifted in frequency by 50 Hz but keeping overall frequency modulations. Speech-recognition thresholds improved when tones with reduced frequency modulation were presented, and improved significantly for noise bands maintaining F0 information. A down-shifted tone, or only a tone to indicate voicing, showed no EAS benefit. These results confirm that the presence of the target's F0 is beneficial for EAS hearing in a noisy environment, and they indicate that the benefit is robust to F0 distortion, as long as the mean F0 and frequency modulations of F0 are preserved.

Discriminability and Perceptual Saliency of Temporal and Spectral Cues for Final Fricative Consonant Voicing in Simulated Cochlear-Implant and Bimodal Hearing

Multiple redundant acoustic cues can contribute to the perception of a single phonemic contrast. This study investigated the effect of spectral degradation on the discriminability and perceptual saliency of acoustic cues for identification of word-final fricative voicing in "loss" versus "laws", and possible changes that occurred when low-frequency acoustic cues were restored. Three acoustic cues that contribute to the word-final /s/-/z/ contrast (first formant frequency [F1] offset, vowel-consonant duration ratio, and consonant voicing duration) were systematically varied in synthesized words. A discrimination task measured listeners' ability to discriminate differences among stimuli within a single cue dimension. A categorization task examined the extent to which listeners make use of a given cue to label a syllable as "loss" versus "laws" when multiple cues are available. Normal-hearing listeners were presented with stimuli that were either unprocessed, processed with an eight-channel noise-band vocoder to approximate spectral degradation in cochlear implants, or low-pass filtered. Listeners were tested in four listening conditions: unprocessed, vocoder, low-pass, and a combined vocoder + low-pass condition that simulated bimodal hearing. Results showed a negative impact of spectral degradation on F1 cue discrimination and a trading relation between spectral and temporal cues in which listeners relied more heavily on the temporal cues for "loss-laws" identification when spectral cues were degraded. Furthermore, the addition of low-frequency fine-structure cues in simulated bimodal hearing increased the perceptual saliency of the F1 cue for "loss-laws" identification compared with vocoded speech. Findings suggest an interplay between the quality of sensory input and cue importance.

The role of continuous low-frequency harmonicity cues for interrupted speech perception in bimodal hearing

Low-frequency acoustic cues have been shown to enhance speech perception by cochlear-implant users, particularly when target speech occurs in a competing background. The present study examined the extent to which a continuous representation of low-frequency harmonicity cues contributes to bimodal benefit in simulated bimodal listeners. Experiment 1 examined the benefit of restoring a continuous temporal envelope to the low-frequency ear while the vocoder ear received a temporally interrupted stimulus. Experiment 2 examined the effect of providing continuous harmonicity cues in the low-frequency ear as compared to restoring a continuous temporal envelope in the vocoder ear. Findings indicate that bimodal benefit for temporally interrupted speech increases when continuity is restored to either or both ears. The primary benefit appears to stem from the continuous temporal envelope in the low-frequency region providing additional phonetic cues related to manner and F1 frequency; a secondary contribution is provided by low-frequency harmonicity cues when a continuous representation of the temporal envelope is present in the low-frequency, or both ears. The continuous temporal envelope and harmonicity cues of low-frequency speech are thought to support bimodal benefit by facilitating identification of word and syllable boundaries, and by restoring partial phonetic cues that occur during gaps in the temporally interrupted stimulus.

Acoustic source characteristics, across-formant integration, and speech intelligibility under competitive conditions

An important aspect of speech perception is the ability to group or select formants using cues in the acoustic source characteristics—for example, fundamental frequency (F0) differences between formants promote their segregation. This study explored the role of more radical differences in source characteristics. Three-formant (F1+F2+F3) synthetic speech analogues were derived from natural sentences. In Experiment 1, F1+F3 were generated by passing a harmonic glottal source (F0 = 140 Hz) through second-order resonators (H1+H3); in Experiment 2, F1+F3 were tonal (sine-wave) analogues (T1+T3). F2 could take either form (H2 or T2). In some conditions, the target formants were presented alone, either monaurally or dichotically (left ear = F1+F3; right ear = F2). In others, they were accompanied by a competitor for F2 (F1+F2C+F3; F2), which listeners must reject to optimize recognition. Competitors (H2C or T2C) were created using the time-reversed frequency and amplitude contours of F2. Dichotic presentation of F2 and F2C ensured that the impact of the competitor arose primarily through informational masking. In the absence of F2C, the effect of a source mismatch between F1+F3 and F2 was relatively modest. When F2C was present, intelligibility was lowest when F2 was tonal and F2C was harmonic, irrespective of which type matched F1+F3. This finding suggests that source type and context, rather than similarity, govern the phonetic contribution of a formant. It is proposed that wideband harmonic analogues are more effective informational maskers than narrowband tonal analogues, and so become dominant in across-frequency integration of phonetic information when placed in competition.

Tone and sentence perception in young Mandarin-speaking children with cochlear implants

OBJECTIVES

The purpose of this study was to examine the outcomes of cochlear implantation in young children in terms of (1) perception of lexical tones in quiet, (2) perception of sentences in quiet and in noise, (3) the effects of five demographic variables (i.e., preoperative hearing level, age at implantation, duration of cochlear implants use, maternal educational level, and whether a child underwent a hearing aid trial before implantation) on lexical tone perception and sentence perception, and (4) the relationship between lexical tone perception and sentence perception.

METHODS

96 participants, aged from 2.41 years to 7.09 years, were recruited in mainland China. The children exhibited normal cognitive abilities and received unilateral implants at an average age of 2.72 years, with ages ranging from 0.69 to 5 years of age.

RESULTS

The mean score for tone identification was 77% (SD=13%; chance level=50%). Tone 2/tone 3 was the most difficult tone contrast to identify. Children with a longer duration of CI use and whose mothers had more years of education tended to perform better in sentence perception in quiet and in noise. Having undergone a hearing aid trial before implantation and more residual hearing were additional factors contributing to better sentence perception in noise. The only demographical variable that related to tone perception in quiet was duration of CI. In addition, while there was a modest correlation between tone perception and sentence perception in quiet (rs=0.47, p<0.001), the correlation between tone perception in quiet and sentence perception in noise was much weaker (rs=-0.28, p<0.05).

CONCLUSIONS

The findings suggested that most young children who had been implanted before 5 years of age and had 1-3 years of implant use did not catch up with their aged peers with normal hearing in tone perception and sentence perception. The weak to moderate correlation between tone perception in quiet and sentence perception might imply that the improvement of tone perception in quiet may not necessarily contribute to sentence perception, especially in noise condition.

Exploring the roles of spectral detail and intonation contour in speech intelligibility: an FMRI study

The melodic contour of speech forms an important perceptual aspect of tonal and nontonal languages and an important limiting factor on the intelligibility of speech heard through a cochlear implant. Previous work exploring the neural correlates of speech comprehension identified a left-dominant pathway in the temporal lobes supporting the extraction of an intelligible linguistic message, whereas the right anterior temporal lobe showed an overall preference for signals clearly conveying dynamic pitch information [Johnsrude, I. S., Penhune, V. B., & Zatorre, R. J. Functional specificity in the right human auditory cortex for perceiving pitch direction. Brain, 123, 155-163, 2000; Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400-2406, 2000]. The current study combined modulations of overall intelligibility (through vocoding and spectral inversion) with a manipulation of pitch contour (normal vs. falling) to investigate the processing of spoken sentences in functional MRI. Our overall findings replicate and extend those of Scott et al. [Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400-2406, 2000], where greater sentence intelligibility was predominately associated with increased activity in the left STS, and the greatest response to normal sentence melody was found in right superior temporal gyrus. These data suggest a spatial distinction between brain areas associated with intelligibility and those involved in the processing of dynamic pitch information in speech. By including a set of complexity-matched unintelligible conditions created by spectral inversion, this is additionally the first study reporting a fully factorial exploration of spectrotemporal complexity and spectral inversion as they relate to the neural processing of speech intelligibility. Perhaps surprisingly, there was little evidence for an interaction between the two factors-we discuss the implications for the processing of sound and speech in the dorsolateral temporal lobes.

The benefits of bimodal hearing: effect of frequency region and acoustic bandwidth

We examined the effects of acoustic bandwidth on bimodal benefit for speech recognition in adults with a cochlear implant (CI) in one ear and low-frequency acoustic hearing in the contralateral ear. The primary aims were to (1) replicate Zhang et al. [Ear Hear 2010;31:63-69] with a steeper filter roll-off to examine the low-pass bandwidth required to obtain bimodal benefit for speech recognition and expand results to include different signal-to-noise ratios (SNRs) and talker genders, (2) determine whether the bimodal benefit increased with acoustic low-pass bandwidth and (3) determine whether an equivalent bimodal benefit was obtained with acoustic signals of similar low-pass and pass band bandwidth, but different center frequencies. Speech recognition was assessed using words presented in quiet and sentences in noise (+10, +5 and 0 dB SNRs). Acoustic stimuli presented to the nonimplanted ear were filtered into the following bands: <125, 125-250, <250, 250-500, <500, 250-750, <750 Hz and wide-band (full, nonfiltered bandwidth). The primary findings were: (1) the minimum acoustic low-pass bandwidth that produced a significant bimodal benefit was <250 Hz for male talkers in quiet and for female talkers in multitalker babble, but <125 Hz for male talkers in background noise, and the observed bimodal benefit did not vary significantly with SNR; (2) the bimodal benefit increased systematically with acoustic low-pass bandwidth up to <750 Hz for a male talker in quiet and female talkers in noise and up to <500 Hz for male talkers in noise, and (3) a similar bimodal benefit was obtained with low-pass and band-pass-filtered stimuli with different center frequencies (e.g. <250 vs. 250-500 Hz), meaning multiple frequency regions contain useful cues for bimodal benefit. Clinical implications are that (1) all aidable frequencies should be amplified in individuals with bimodal hearing, and (2) verification of audibility at 125 Hz is unnecessary unless it is the only aidable frequency.

Design and evaluation of a personal digital assistant-based research platform for cochlear implants

This paper discusses the design, development, features, and clinical evaluation of a personal digital assistant (PDA)-based platform for cochlear implant research. This highly versatile and portable research platform allows researchers to design and perform complex experiments with cochlear implants manufactured by Cochlear Corporation with great ease and flexibility. The research platform includes a portable processor for implementing and evaluating novel speech processing algorithms, a stimulator unit which can be used for electrical stimulation and neurophysiologic studies with animals, and a recording unit for collecting electroencephalogram/evoked potentials from human subjects. The design of the platform for real time and offline stimulation modes is discussed for electric-only and electric plus acoustic stimulation followed by results from an acute study with implant users for speech intelligibility in quiet and noisy conditions. The results are comparable with users' clinical processor and very promising for undertaking chronic studies.

The organization and physiology of the auditory thalamus and its role in processing acoustic features important for speech perception

The auditory thalamus, or medial geniculate body (MGB), is the primary sensory input to auditory cortex. Therefore, it plays a critical role in the complex auditory processing necessary for robust speech perception. This review will describe the functional organization of the thalamus as it relates to processing acoustic features important for speech perception, focusing on thalamic nuclei that relate to auditory representations of language sounds. The MGB can be divided into three main subdivisions, the ventral, dorsal, and medial subdivisions, each with different connectivity, auditory response properties, neuronal properties, and synaptic properties. Together, the MGB subdivisions actively and dynamically shape complex auditory processing and form ongoing communication loops with auditory cortex and subcortical structures.

Speech perception in noise by children with cochlear implants

PURPOSE

Common wisdom suggests that listening in noise poses disproportionately greater difficulty for listeners with cochlear implants (CIs) than for peers with normal hearing (NH). The purpose of this study was to examine phonological, language, and cognitive skills that might help explain speech-in-noise abilities for children with CIs.

METHOD

Three groups of kindergartners (NH, hearing aid wearers, and CI users) were tested on speech recognition in quiet and noise and on tasks thought to underlie the abilities that fit into the domains of phonological awareness, general language, and cognitive skills. These last measures were used as predictor variables in regression analyses with speech-in-noise scores as dependent variables.

RESULTS

Compared to children with NH, children with CIs did not perform as well on speech recognition in noise or on most other measures, including recognition in quiet. Two surprising results were that (a) noise effects were consistent across groups and (b) scores on other measures did not explain any group differences in speech recognition.

CONCLUSIONS

Limitations of implant processing take their primary toll on recognition in quiet and account for poor speech recognition and language/phonological deficits in children with CIs. Implications are that teachers/clinicians need to teach language/phonology directly and maximize signal-to-noise levels in the classroom.

Beneficial acoustic speech cues for cochlear implant users with residual acoustic hearing

This study investigated which acoustic cues within the speech signal are responsible for bimodal speech perception benefit. Seven cochlear implant (CI) users with usable residual hearing at low frequencies in the non-implanted ear participated. Sentence tests were performed in near-quiet (some noise on the CI side to reduce scores from ceiling) and in a modulated noise background, with the implant alone and with the addition, in the hearing ear, of one of four types of acoustic signals derived from the same sentences: (1) a complex tone modulated by the fundamental frequency (F0) and amplitude envelope contours; (2) a pure tone modulated by the F0 and amplitude contours; (3) a noise-vocoded signal; (4) unprocessed speech. The modulated tones provided F0 information without spectral shape information, whilst the vocoded signal presented spectral shape information without F0 information. For the group as a whole, only the unprocessed speech condition provided significant benefit over implant-alone scores, in both near-quiet and noise. This suggests that, on average, F0 or spectral cues in isolation provided limited benefit for these subjects in the tested listening conditions, and that the significant benefit observed in the full-signal condition was derived from implantees' use of a combination of these cues.

The relative phonetic contributions of a cochlear implant and residual acoustic hearing to bimodal speech perception

The addition of low-passed (LP) speech or even a tone following the fundamental frequency (F0) of speech has been shown to benefit speech recognition for cochlear implant (CI) users with residual acoustic hearing. The mechanisms underlying this benefit are still unclear. In this study, eight bimodal subjects (CI users with acoustic hearing in the non-implanted ear) and eight simulated bimodal subjects (using vocoded and LP speech) were tested on vowel and consonant recognition to determine the relative contributions of acoustic and phonetic cues, including F0, to the bimodal benefit. Several listening conditions were tested (CI/Vocoder, LP, T(F0-env), CI/Vocoder + LP, CI/Vocoder + T(F0-env)). Compared with CI/Vocoder performance, LP significantly enhanced both consonant and vowel perception, whereas a tone following the F0 contour of target speech and modulated with an amplitude envelope of the maximum frequency of the F0 contour (T(F0-env)) enhanced only consonant perception. Information transfer analysis revealed a dual mechanism in the bimodal benefit: The tone representing F0 provided voicing and manner information, whereas LP provided additional manner, place, and vowel formant information. The data in actual bimodal subjects also showed that the degree of the bimodal benefit depended on the cutoff and slope of residual acoustic hearing.