Speech Perception with Cochlear Implants

Lexical Stress Identification in Cochlear Implant-Simulated Speech by Non-Native Listeners

This study investigates whether a presumed difference in the perceptibility of cues to lexical stress in spectro-temporally degraded simulated cochlear implant (CI) speech affects how listeners weight these cues during a lexical stress identification task, specifically in their non-native language. Previous research suggests that in English, listeners predominantly rely on a reduction in vowel quality as a cue to lexical stress. In Dutch, changes in the fundamental frequency (F0) contour seem to have a greater functional weight than the vowel quality contrast. Generally, non-native listeners use the cue-weighting strategies from their native language in the non-native language. Moreover, few studies have suggested that these cues to lexical stress are differently perceptible in spectro-temporally degraded electric hearing, as CI users appear to make more effective use of changes in vowel quality than of changes in the F0 contour as cues to linguistic phenomena. In this study, native Dutch learners of English identified stressed syllables in CI-simulated and non-CI-simulated Dutch and English words that contained changes in the F0 contour and vowel quality as cues to lexical stress. The results indicate that neither the cue-weighting strategies in the native language nor in the non-native language are influenced by the perceptibility of cues in the spectro-temporally degraded speech signal. These results are in contrast to our expectations based on previous research and support the idea that cue weighting is a flexible and transferable process.

Prosodic Focus Interpretation in Spectrotemporally Degraded Speech by Non-Native Listeners

PURPOSE

This study assesses how spectrotemporal degradations that can occur in the sound transmission of a cochlear implant (CI) may influence the ability of non-native listeners to recognize the intended meaning of utterances based on the position of the prosodically focused word. Previous research suggests that perceptual accuracy and listening effort are negatively affected by CI processing (or CI simulations) or when the speech is presented in a non-native language, in a number of tasks and circumstances. How these two factors interact to affect prosodic focus interpretation, however, remains unclear.

METHOD

In an online experiment, normal-hearing (NH) adolescent and adult native Dutch learners of English and a small control group of NH native English adolescents listened to CI-simulated (eight-channel noise-band vocoded) and non-CI-simulated English sentences differing in prosodically marked focus. For assessing perceptual accuracy, listeners had to indicate which of four possible context questions the speaker answered. For assessing listening effort, a dual-task paradigm was used with a secondary free recall task.

RESULTS

The results indicated that prosodic focus interpretation was significantly less accurate in the CI-simulated condition compared with the non-CI-simulated condition but that listening effort was not increased. Moreover, there was no interaction between the influence of the degraded CI-simulated speech signal and listening groups in either their perceptual accuracy or listening effort.

CONCLUSION

Non-native listeners are not more strongly affected by spectrotemporal degradations than native listeners, and less proficient non-native listeners are not more strongly affected by these degradations than more proficient non-native listeners.

Combining Place and Rate of Stimulation Improves Frequency Discrimination in Cochlear Implant Users

In the auditory system, frequency is represented as tonotopic and temporal response properties of the auditory nerve. While these response properties are inextricably linked in normal hearing, cochlear implants can separately excite tonotopic location and temporal synchrony using different electrodes and stimulation rates, respectively. This separation allows for the investigation of the contributions of tonotopic and temporal cues for frequency discrimination. The present study examines frequency discrimination in adult cochlear implant users as conveyed by electrode position and stimulation rate, separately and combined. The working hypothesis is that frequency discrimination is better provided by place and rate cues combined compared to either cue alone. This hypothesis was tested in two experiments. In the first experiment, frequency discrimination needed for melodic contour identification was measured for frequencies near 100, 200, and 400 Hz using frequency allocation modeled after clinical processors. In the second experiment, frequency discrimination for pitch ranking was measured for frequencies between 100 and 1600 Hz using an experimental frequency allocation designed to provide better access to place cues. The results of both experiments indicate that frequency discrimination is better with place and rate cues combined than with either cue alone. These results clarify how signal processing for cochlear implants could better encode frequency into place and rate of electrical stimulation. Further, the results provide insight into the contributions of place and rate cues for pitch.

Computer-based musical interval training program for Cochlear implant users and listeners with no known hearing loss

A musical interval is the difference in pitch between two sounds. The way that musical intervals are used in melodies relative to the tonal center of a key can strongly affect the emotion conveyed by the melody. The present study examines musical interval identification in people with no known hearing loss and in cochlear implant users. Pitch resolution varies widely among cochlear implant users with average resolution an order of magnitude worse than in normal hearing. The present study considers the effect of training on musical interval identification and tests for correlations between low-level psychophysics and higher-level musical abilities. The overarching hypothesis is that cochlear implant users are limited in their ability to identify musical intervals both by low-level access to frequency cues for pitch as well as higher-level mapping of the novel encoding of pitch that implants provide. Participants completed a 2-week, online interval identification training. The benchmark tests considered before and after interval identification training were pure tone detection thresholds, pure tone frequency discrimination, fundamental frequency discrimination, tonal and rhythm comparisons, and interval identification. The results indicate strong correlations between measures of pitch resolution with interval identification; however, only a small effect of training on interval identification was observed for the cochlear implant users. Discussion focuses on improving access to pitch cues for cochlear implant users and on improving auditory training for musical intervals.

The development of auditory temporal processing during the first year of life

OBJECTIVES

The processing of auditory temporal information is important for the extraction of voice pitch, linguistic information, as well as the overall temporal structure of speech. However, many aspects of its early development remain poorly understood. This paper reviews the development of auditory temporal processing during the first year of life when infants are acquiring their native language.

METHODS

First, potential mechanisms of neural immaturity are discussed in the context of neurophysiological studies. Next, what is known about infant auditory capabilities is considered with a focus on psychophysical studies involving non-speech stimuli to investigate the perception of temporal fine structure and envelope cues. This is followed by a review of studies involving speech stimuli, including those that present vocoded signals as a method of degrading the spectro-temporal information available to infant listeners.

RESULTS/CONCLUSION

This review suggests that temporal resolution may be well developed in the first postnatal months, but that the ability to use and process the temporal information in an efficient way along the entire auditory pathway is longer to develop. Those findings have crucial implications for the development of language abilities, especially for infants with hearing impairment who are using cochlear implants.

Channel Interaction During Infrared Light Stimulation in the Cochlea

BACKGROUND AND OBJECTIVES

The number of perceptually independent channels to encode acoustic information is limited in contemporary cochlear implants (CIs) because of the current spread in the tissue. It has been suggested that neighboring electrodes have to be separated in humans by a distance of more than 2 mm to eliminate significant overlap of the electric current fields and subsequent interaction between the channels. It has also been argued that an increase in the number of independent channels could improve CI user performance in challenging listening environments, such as speech in noise, tonal languages, or music perception. Optical stimulation has been suggested as an alternative modality for neural stimulation because it is spatially selective. This study reports the results of experiments designed to quantify the interaction between neighboring optical sources in the cochlea during stimulation with infrared radiation.

STUDY DESIGN/MATERIALS AND METHODS

In seven adult albino guinea pigs, a forward masking method was used to quantify the interaction between two neighboring optical sources during stimulation. Two optical fibers were placed through cochleostomies into the scala tympani of the basal cochlear turn. The radiation beams were directed towards different neuron populations along the spiral ganglion. Optically evoked compound action potentials were recorded for different radiant energies and distances between the optical fibers. The outcome measure was the radiant energy of a masker pulse delivered 3 milliseconds before a probe pulse to reduce the response evoked by the probe pulse by 3 dB. Results were compared for different distances between the fibers placed along the cochlea.

RESULTS

The energy required to reduce the probe's response by 3 dB increased by 20.4 dB/mm and by 26.0 dB/octave. The inhibition was symmetrical for the masker placed basal to the probe (base-to-apex) and the masker placed apical to the probe (apex-to-base).

CONCLUSION

The interaction between neighboring optical sources during infrared laser stimulation is less than the interaction between neighboring electrical contacts during electrical stimulation. Previously published data for electrical stimulation reported an average current spread in human and cat cochleae of 2.8 dB/mm. With the increased number of independent channels for optical stimulation, it is anticipated that speech and music performance will improve. Lasers Surg. Med. © 2020 Wiley Periodicals LLC.

Lateralized Cerebral Processing of Abstract Linguistic Structure in Clear and Degraded Speech

Human cortical activity measured with magnetoencephalography (MEG) has been shown to track the temporal regularity of linguistic information in connected speech. In the current study, we investigate the underlying neural sources of these responses and test the hypothesis that they can be directly modulated by changes in speech intelligibility. MEG responses were measured to natural and spectrally degraded (noise-vocoded) speech in 19 normal hearing participants. Results showed that cortical coherence to "abstract" linguistic units with no accompanying acoustic cues (phrases and sentences) were lateralized to the left hemisphere and changed parametrically with intelligibility of speech. In contrast, responses coherent to words/syllables accompanied by acoustic onsets were bilateral and insensitive to intelligibility changes. This dissociation suggests that cerebral responses to linguistic information are directly affected by intelligibility but also powerfully shaped by physical cues in speech. This explains why previous studies have reported widely inconsistent effects of speech intelligibility on cortical entrainment and, within a single experiment, provided clear support for conclusions about language lateralization derived from a large number of separately conducted neuroimaging studies. Since noise-vocoded speech resembles the signals provided by a cochlear implant device, the current methodology has potential clinical utility for assessment of cochlear implant performance.

Perceptual learning of pitch provided by cochlear implant stimulation rate

Cochlear implant users hear pitch evoked by stimulation rate, but discrimination diminishes for rates above 300 Hz. This upper limit on rate pitch is surprising given the remarkable and specialized ability of the auditory nerve to respond synchronously to stimulation rates at least as high as 3 kHz and arguably as high as 10 kHz. Sensitivity to stimulation rate as a pitch cue varies widely across cochlear implant users and can be improved with training. The present study examines individual differences and perceptual learning of stimulation rate as a cue for pitch ranking. Adult cochlear implant users participated in electrode psychophysics that involved testing once per week for three weeks. Stimulation pulse rate discrimination was measured in bipolar and monopolar configurations for apical and basal electrodes. Base stimulation rates between 100 and 800 Hz were examined. Individual differences were quantified using psychophysically derived metrics of spatial tuning and temporal integration. This study examined distribution of measures across subjects, predictive power of psychophysically derived metrics of spatial tuning and temporal integration, and the effect of training on rate discrimination thresholds. Psychophysical metrics of spatial tuning and temporal integration were not predictive of stimulation rate discrimination, but discrimination thresholds improved at lower frequencies with training. Since most clinical devices do not use variable stimulation rates, it is unknown to what extent recipients may learn to use stimulation rate cues if provided in a clear and consistent manner.

Does training with amplitude modulated tones affect tone-vocoded speech perception?

Temporal-envelope cues are essential for successful speech perception. We asked here whether training on stimuli containing temporal-envelope cues without speech content can improve the perception of spectrally-degraded (vocoded) speech in which the temporal-envelope (but not the temporal fine structure) is mainly preserved. Two groups of listeners were trained on different amplitude-modulation (AM) based tasks, either AM detection or AM-rate discrimination (21 blocks of 60 trials during two days, 1260 trials; frequency range: 4Hz, 8Hz, and 16Hz), while an additional control group did not undertake any training. Consonant identification in vocoded vowel-consonant-vowel stimuli was tested before and after training on the AM tasks (or at an equivalent time interval for the control group). Following training, only the trained groups showed a significant improvement in the perception of vocoded speech, but the improvement did not significantly differ from that observed for controls. Thus, we do not find convincing evidence that this amount of training with temporal-envelope cues without speech content provide significant benefit for vocoded speech intelligibility. Alternative training regimens using vocoded speech along the linguistic hierarchy should be explored.

Neuronal Development of Hearing and Language: Cochlear Implants and Critical Periods

The modern cochlear implant (CI) is the most successful neural prosthesis developed to date. CIs provide hearing to the profoundly hearing impaired and allow the acquisition of spoken language in children born deaf. Results from studies enabled by the CI have provided new insights into ( a) minimal representations at the periphery for speech reception, ( b) brain mechanisms for decoding speech presented in quiet and in acoustically adverse conditions, ( c) the developmental neuroscience of language and hearing, and ( d) the mechanisms and time courses of intramodal and cross-modal plasticity. Additionally, the results have underscored the interconnectedness of brain functions and the importance of top-down processes in perception and learning. The findings are described in this review with emphasis on the developing brain and the acquisition of hearing and spoken language.

Speech perception in children with cochlear implants for continua varying in formant transition duration

PURPOSE

To examine the developmental course of labial and alveolar manner of articulation contrasts, and to determine how that course may be different for typically developing (TD) children with cochlear implants (CI).

METHOD

Eight young adults, eight TD 5-8 year-old children, and seven 5-8 year-old children with CIs participated. Labial /ba/-/wa/ and alveolar /da/-/ja/ continua stimuli were presented, with each continuum consisting of nine synthetic stimuli varying in F2 and F3 transition duration. Participants were asked to label the stimuli as either a stop or glide, and responses were analysed for phonetic boundaries and slopes.

RESULT

For the /ba/-/wa/ contrast, children with CIs required longer transition durations compared to TD children or adults to cross from one phoneme category to another. The children with CIs demonstrated less confidence in labelling the stimuli (i.e. less steep slopes) than the TD children or the adults. For the /da/-/ja/ contrast, the children with CIs showed less steep slope values than adults.

CONCLUSION

These results suggest that there are differences in the way TD children and children with CIs develop and maintain phonetic categories, perhaps differences in phonetic representation or in linking acoustic and phonetic representations.

Combined Effects of Noise and Reverberation on Sound Localization for Listeners With Normal Hearing and Bilateral Cochlear Implants

PURPOSE

This study examined the individual and combined effects of noise and reverberation on the ability of listeners with normal hearing (NH) and with bilateral cochlear implants (BCIs) to localize speech.

METHOD

Six adults with BCIs and 10 with NH participated. All subjects completed a virtual localization test in quiet and at 0-, -4-, and -8-dB signal-to-noise ratios (SNRs) in simulated anechoic and reverberant (0.2-, 0.6-, and 0.9-s RT60) environments. BCI users were also tested at +8- and +4-dB SNR. A 3-word phrase was presented at 70 dB SPL from 9 simulated locations in the frontal horizontal plane (±90°), with the noise source at 0°.

RESULTS

BCIs users had significantly poorer localization than listeners with NH in all conditions. BCI users' performance started to decrease at a higher SNR (+4 dB) and shorter RT60 (0.2 s) than listeners with NH (-4 dB and 0.6 s). The combination of noise and reverberation began to degrade localization of BCI users at a higher SNR and a shorter RT60 than listeners with NH.

CONCLUSION

The clear effect of noise and reverberation on the performance of BCI users provides information that should be useful for refining cochlear implant processing strategies and developing cochlear implant rehabilitation plans to optimize binaural benefit for BCI users in everyday listening situations.

Alveolar and Postalveolar Voiceless Fricative and Affricate Productions of Spanish-English Bilingual Children With Cochlear Implants

PURPOSE

This study investigates the production of voiceless alveolar and postalveolar fricatives and affricates by bilingual and monolingual children with hearing loss who use cochlear implants (CIs) and their peers with normal hearing (NH).

METHOD

Fifty-four children participated in our study, including 12 Spanish-English bilingual CI users (M = 6;0 [years;months]), 12 monolingual English-speaking children with CIs (M = 6;1), 20 bilingual children with NH (M = 6;5), and 10 monolingual English-speaking children with NH (M = 5;10). Picture elicitation targeting /s/, /tʃ/, and /ʃ/ was administered. Repeated-measures analyses of variance comparing group means for frication duration, rise time, and centroid frequency were conducted for the effects of CI use and bilingualism.

RESULTS

All groups distinguished the target sounds in the 3 acoustic parameters examined. Regarding frication duration and rise time, the Spanish productions of bilingual children with CIs differed from their bilingual peers with NH. English frication duration patterns for bilingual versus monolingual CI users also differed. Centroid frequency was a stronger place cue for children with NH than for children with CIs.

CONCLUSION

Patterns of fricative and affricate production display effects of bilingualism and diminished signal, yielding unique patterns for bilingual and monolingual CI users.

The Effects of Phonotactic Probability and Neighborhood Density on Adults' Word Learning in Noisy Conditions

PURPOSE

Noisy conditions make auditory processing difficult. This study explores whether noisy conditions influence the effects of phonotactic probability (the likelihood of occurrence of a sound sequence) and neighborhood density (phonological similarity among words) on adults' word learning.

METHOD

Fifty-eight adults learned nonwords varying in phonotactic probability and neighborhood density in either an unfavorable (0-dB signal-to-noise ratio [SNR]) or a favorable (+8-dB SNR) listening condition. Word learning was assessed using a picture naming task by scoring the proportion of phonemes named correctly.

RESULTS

The unfavorable 0-dB SNR condition showed a significant interaction between phonotactic probability and neighborhood density in the absence of main effects. In particular, adults learned more words when phonotactic probability and neighborhood density were both low or both high. The +8-dB SNR condition did not show this interaction. These results are inconsistent with those from a prior adult word learning study conducted under quiet listening conditions that showed main effects of word characteristics.

CONCLUSIONS

As the listening condition worsens, adult word learning benefits from a convergence of phonotactic probability and neighborhood density. Clinical implications are discussed for potential populations who experience difficulty with auditory perception or processing, making them more vulnerable to noise.

Role of contextual cues on the perception of spectrally reduced interrupted speech

Understanding speech within an auditory scene is constantly challenged by interfering noise in suboptimal listening environments when noise hinders the continuity of the speech stream. In such instances, a typical auditory-cognitive system perceptually integrates available speech information and "fills in" missing information in the light of semantic context. However, individuals with cochlear implants (CIs) find it difficult and effortful to understand interrupted speech compared to their normal hearing counterparts. This inefficiency in perceptual integration of speech could be attributed to further degradations in the spectral-temporal domain imposed by CIs making it difficult to utilize the contextual evidence effectively. To address these issues, 20 normal hearing adults listened to speech that was spectrally reduced and spectrally reduced interrupted in a manner similar to CI processing. The Revised Speech Perception in Noise test, which includes contextually rich and contextually poor sentences, was used to evaluate the influence of semantic context on speech perception. Results indicated that listeners benefited more from semantic context when they listened to spectrally reduced speech alone. For the spectrally reduced interrupted speech, contextual information was not as helpful under significant spectral reductions, but became beneficial as the spectral resolution improved. These results suggest top-down processing facilitates speech perception up to a point, and it fails to facilitate speech understanding when the speech signals are significantly degraded.

The influence of word characteristics on the vocabulary of children with cochlear implants

The goal of this study was to explore the effects of phonotactic probability, word length, word frequency, and neighborhood density on the words known by children with cochlear implants (CIs) varying in vocabulary outcomes in a retrospective analysis of a subset of data from a longitudinal study of hearing loss. Generalized linear mixed modeling was used to examine the effects of these word characteristics at 3 time points: preimplant, postimplant, and longitudinal follow-up. Results showed a robust effect of neighborhood density across group and time, whereas the effect of frequency varied by time. Significant effects of phonotactic probability or word length were not detected. Taken together, these findings suggest that children with CIs may be able to use spoken language structure in a manner similar to their normal hearing counterparts, despite the differences in the quality of the input. The differences in the effects of phonotactic probability and word length imply a difficulty in initiating word learning and limited working memory ability in children with CIs.

Effects of simulated spectral holes on speech intelligibility and spatial release from masking under binaural and monaural listening

The possibility that "dead regions" or "spectral holes" can account for some differences in performance between bilateral cochlear implant (CI) users and normal-hearing listeners was explored. Using a 20-band noise-excited vocoder to simulate CI processing, this study examined effects of spectral holes on speech reception thresholds (SRTs) and spatial release from masking (SRM) in difficult listening conditions. Prior to processing, stimuli were convolved through head-related transfer-functions to provide listeners with free-field directional cues. Processed stimuli were presented over headphones under binaural or monaural (right ear) conditions. Using Greenwood's [(1990). J. Acoust. Soc. Am. 87, 2592-2605] frequency-position function and assuming a cochlear length of 35 mm, spectral holes were created for variable sizes (6 and 10 mm) and locations (base, middle, and apex). Results show that middle-frequency spectral holes were the most disruptive to SRTs, whereas high-frequency spectral holes were the most disruptive to SRM. Spectral holes generally reduced binaural advantages in difficult listening conditions. These results suggest the importance of measuring dead regions in CI users. It is possible that customized programming for bilateral CI processors based on knowledge about dead regions can enhance performance in adverse listening situations.

Forward-masking patterns produced by symmetric and asymmetric pulse shapes in electric hearing

Two forward-masking experiments were conducted with six cochlear implant listeners to test whether asymmetric pulse shapes would improve the place-specificity of stimulation compared to symmetric ones. The maskers were either cathodic-first symmetric biphasic, pseudomonophasic (i.e., with a second anodic phase longer and lower in amplitude than the first phase), or "delayed pseudomonophasic" (identical to pseudomonophasic but with an inter-phase gap) stimuli. In experiment 1, forward-masking patterns for monopolar maskers were obtained by keeping each masker fixed on a middle electrode of the array and measuring the masked thresholds of a monopolar signal presented on several other electrodes. The results were very variable, and no difference between pulse shapes was found. In experiment 2, six maskers were used in a wide bipolar (bipolar+9) configuration: the same three pulse shapes as in experiment 1, either cathodic-first relative to the most apical or relative to the most basal electrode of the bipolar channel. The pseudomonophasic masker showed a stronger excitation proximal to the electrode of the bipolar pair for which the short, high-amplitude phase was anodic. However, no difference was obtained with the symmetric and, more surprisingly, with the delayed pseudomonophasic maskers. Implications for cochlear implant design are discussed.

Music in the lives of deaf children with cochlear implants

Present-day cochlear implants provide good temporal cues and coarse spectral cues. In general, these cues are adequate for perceiving speech in quiet backgrounds and for young children's acquisition of spoken language. They are inadequate, however, for conveying the rich pitch-patterning of music. As a result, many adults who become implant users after losing their hearing find music disappointing or unacceptable. By contrast, child implant users who were born deaf or became deaf as infants or toddlers typically find music interesting and enjoyable. They recognize popular songs that they hear regularly when the test materials match critical features of the original versions. For example, they can identify familiar songs from the original recordings with words and from versions that omit the words but preserve all other cues. They also recognize theme songs from their favorite television programs when presented in original or somewhat altered form. The motivation of children with implants for listening to music or melodious speech is evident well before they understand language. Within months after receiving their implant, they prefer singing to silence. They also prefer speech in the maternal style to typical adult speech and the sounds of their native language-to-be to those of a foreign language. An important task of future research is to ascertain the relative contributions of perceptual and motivational factors to the apparent differences between child and adult implant users.

Unilateral and bilateral cochlear implants and the implant-plus-hearing-aid profile: comparing self-assessed and measured abilities

Patients fitted with one (CI) versus two (CI+CI) cochlear implants, and those fitted with one implant who retain a hearing aid in the non-implanted ear (CI+HA), were compared using the speech, spatial, and qualities of hearing scale (SSQ) (Gatehouse & Noble, 2004). The CI+CI profile yielded significantly higher ability ratings than the CI profile in the spatial hearing domain, and on most aspects of other qualities of hearing (segregation, naturalness, and listening effort). A subset of patients completed the SSQ prior to implantation, and the CI+CI profile showed consistently greater improvement than the CI profile across all domains. Patients in the CI+HA group self-rated no differently from the CI group, post-implant. Measured speech perception and localization performance showed some parallels with the self-rating outcomes. Overall, a unilateral CI provided significant benefit across most hearing functions reflected in the SSQ. Bilateral implantation offered further benefit across a substantial range of those functions.

Some benefits and limitations of binaural cochlear implants and our ability to measure them

We review new recognition and localization skills in patients using one or two cochlear implant(s). We observed one unilateral patient who showed localization performance above chance. We also provide evidence for binaural processing in bilateral cochlear implant patients, even when tested with speech from the front without noise. We unsuccessfully attempted to find correlations between localization and squelch, between these variables and pre-implant threshold differences, or these variables and post-implant recognition differences. We strongly believe that new tests are needed to examine the potential benefit of two implants. We describe three tests that we use to show a binaural advantage: cued recognition, movement direction, and recognition with multiple jammers.