Adaptation by normal listeners to upward spectral shifts of speech: implications for cochlear implants

Interactions between unsupervised learning and the degree of spectral mismatch on short-term perceptual adaptation to spectrally shifted speech

OBJECTIVES

Cochlear implant listeners are able to at least partially adapt to the spectral mismatch associated with the implant device and speech processor via daily exposure and/or explicit training. The overall goal of this study was to investigate interactions between short-term unsupervised learning (i.e., passive adaptation) and the degree of spectral mismatch in normal-hearing listeners' adaptation to spectrally shifted vowels.

DESIGN

Normal-hearing subjects were tested while listening to acoustic cochlear implant simulations. Unsupervised learning was measured by testing vowel recognition repeatedly over a 5 day period; no feedback or explicit training was provided. In experiment 1, subjects listened to 8-channel, sine-wave vocoded speech. The spectral envelope was compressed to simulate a 16 mm cochlear implant electrode array. The analysis bands were fixed and the compressed spectral envelope was linearly shifted toward the base by 3.6, 6, or 8.3 mm to simulate different insertion depths of the electrode array, resulting in a slight, moderate, or severe spectral shift. In experiment 2, half the subjects were exclusively exposed to a severe shift with 8 or 16 channels (exclusive groups), and half the subjects were exposed to 8-channel severely shifted speech, 16-channel severely shifted speech, and 8-channel moderately shifted speech, alternately presented within each test session (mixed group). The region of stimulation in the cochlea was fixed (16 mm in extent and 15 mm from the apex) and the analysis bands were manipulated to create the spectral shift conditions. To determine whether increased spectral resolution would improve adaptation, subjects were exposed to 8- or 16-channel severely shifted speech.

RESULTS

In experiment 1, at the end of the adaptation period, there was no significant difference between 8-channel speech that was spectrally matched and that shifted by 3.6 mm. There was a significant, but less-complete, adaptation to the 6 mm shift and no adaptation to the 8.3 mm shift. In experiment 2, for the mixed exposure group, there was significant adaptation to severely shifted speech with 8 channels and even greater adaptation with 16 channels. For the exclusive exposure group, there was no significant adaptation to severely shifted speech with either 8 or 16 channels.

CONCLUSIONS

These findings suggest that listeners are able to passively adapt to spectral shifts up to 6 mm. For spectral shifts beyond 6 mm, some passive adaptation was observed with mixed exposure to a smaller spectral shift, even at the expense of some low frequency information. Mixed exposure to the smaller shift may have enhanced listeners' access to spectral envelope details that were not accessible when listening exclusively to severely shifted speech. The results suggest that the range of spectral mismatch that can support passive adaptation may be larger than previously reported. Some amount of passive adaptation may be possible with severely shifted speech by exposing listeners to a relatively small mismatch in conjunction with the severe mismatch.

Effects of spectral shifting on speech perception in noise

The present study used eight normal-hearing (NH) subjects, listening to acoustic cochlear implant (CI) simulations, to examine the effects of spectral shifting on speech recognition in noise. Speech recognition was measured using spectrally matched and shifted speech (vowels, consonants, and IEEE sentences), generated by 8-channel, sine-wave vocoder. Measurements were made in quiet and in noise (speech-shaped static noise and speech-babble at 5 dB signal-to-noise ratio). One spectral match condition and four spectral shift conditions were investigated: 2 mm, 3 mm, and 4 mm linear shift, and 3 mm shift with compression, in terms of cochlear distance. Results showed that speech recognition scores dropped because of noise and spectral shifting, and that the interactive effects of spectral shifting and background conditions depended on the degree/type of spectral shift, background conditions, and the speech test materials. There was no significant interaction between spectral shifting and two noise conditions for all speech test materials. However, significant interactions between linear spectral shifts and all background conditions were found in sentence recognition; significant interactions between spectral shift types and all background conditions were found in vowel recognition. Overall, the results suggest that tonotopic mismatch may affect performance of CI users in complex listening environments.

The effects of frequency-place shift on consonant confusion in cochlear implant simulations

The effects of frequency-place shift on consonant recognition and confusion matrices were examined. Frequency-place shift was manipulated using a noise-excited vocoder with 4 to 16 channels. In the vocoder processing, the location of the most apical carrier band varied from the matched condition (i.e., 28 mm from the base of the cochlear) to a basal shift (i.e., 22 mm from the base) in a step size of 1 mm. Ten normal-hearing subjects participated in the 20-alternative forced-choice test, where the consonants were presented in a /Ca/ context. Shift of 3 mm or more caused the consonant recognition scores to decrease significantly. The effects of spectral resolution disappeared when the amount of shift reached >or=3 mm. Information transmitted for voicing and place of articulation varied with spectral shift and spectral resolution, while information transmitted for manner was affected only by spectral shift but not spectral resolution. Spectral shift has shown specific effects on the confusion patterns of the consonants. The direction of errors reversed as spectral shift increased and the patterns of reversal were consistent across channel conditions. Overall, transmission of the consonant features can be accounted for by the acoustic features of the speech signal.

Inferior frontal gyrus activation predicts individual differences in perceptual learning of cochlear-implant simulations

This study investigated the neural plasticity associated with perceptual learning of a cochlear implant (CI) simulation. Normal-hearing listeners were trained with vocoded and spectrally shifted speech simulating a CI while cortical responses were measured with functional magnetic resonance imaging (fMRI). A condition in which the vocoded speech was spectrally inverted provided a control for learnability and adaptation. Behavioral measures showed considerable individual variability both in the ability to learn to understand the degraded speech, and in phonological working memory capacity. Neurally, left-lateralized regions in superior temporal sulcus and inferior frontal gyrus (IFG) were sensitive to the learnability of the simulations, but only the activity in prefrontal cortex correlated with interindividual variation in intelligibility scores and phonological working memory. A region in left angular gyrus (AG) showed an activation pattern that reflected learning over the course of the experiment, and covariation of activity in AG and IFG was modulated by the learnability of the stimuli. These results suggest that variation in listeners' ability to adjust to vocoded and spectrally shifted speech is partly reflected in differences in the recruitment of higher-level language processes in prefrontal cortex, and that this variability may further depend on functional links between the left inferior frontal gyrus and angular gyrus. Differences in the engagement of left inferior prefrontal cortex, and its covariation with posterior parietal areas, may thus underlie some of the variation in speech perception skills that have been observed in clinical populations of CI users.

Resistance to learning binaurally mismatched frequency-to-place maps: implications for bilateral stimulation with cochlear implants

Simulations of monaural cochlear implants in normal hearing listeners have shown that the deleterious effects of upward spectral shifting on speech perception can be overcome with training. This study simulates bilateral stimulation with a unilateral spectral shift to investigate whether listeners can adapt to upward-shifted speech information presented together with contralateral unshifted information. A six-channel, dichotic, interleaved sine-carrier vocoder simulated a binaurally mismatched frequency-to-place map. Odd channels were presented to one ear with an upward frequency shift equivalent to 6 mm on the basilar membrane, while even channels were presented to the contralateral ear unshifted. In Experiment 1, listeners were trained for 5.3 h with either the binaurally mismatched processor or with just the shifted monaural bands. In Experiment 2, the duration of training was 10 h, and the trained condition alternated between those of Experiment 1. While listeners showed learning in both experiments, intelligibility with the binaurally mismatched processor never exceeded, intelligibility with just the three unshifted bands, suggesting that listeners did not benefit from combining the mismatched maps, even though there was clear scope to do so. Frequency-place map alignment may thus be of importance when optimizing bilateral devices of the type studied here.

Relative importance of different spectral bands to consonant identification: relevance for frequency transposition in hearing aids

Listeners with high-frequency dead regions (DRs) benefit from amplification of frequencies up to 1.7 times the edge frequency, f(e), of the DR. Better consonant identification might be achieved by replacing the band from f(e) to 1.7f(e) with a higher spectral band. We aimed to identify the optimal band, using simulations with normal-hearing listeners. In experiment 1, nonsense syllables were lowpass filtered to simulate DRs with f(e) of 0.5, 0.75, and 1.0 kHz. Identification was measured for each of these base bands alone and with a bandpass-filtered band added (but not transposed). The added band either extended from f(e) to 1.7f(e) or its center frequency was increased, keeping bandwidth fixed in ERB(N)-number. Performance improved with increasing center frequency and then reached an asymptote or declined. Experiment 2 used a mid-frequency base band, and a lower-frequency added band. The results also showed a beneficial effect of frequency separation of the added and base bands. Experiment 3 resembled experiment 1, but with bandwidth fixed in Hertz. For higher-frequency added bands, the benefit was lower than for experiment 1.

Transfer of auditory perceptual learning with spectrally reduced speech to speech and nonspeech tasks: implications for cochlear implants

OBJECTIVE

The objective of this study was to assess whether training on speech processed with an eight-channel noise vocoder to simulate the output of a cochlear implant would produce transfer of auditory perceptual learning to the recognition of nonspeech environmental sounds, the identification of speaker gender, and the discrimination of talkers by voice.

DESIGN

Twenty-four normal-hearing subjects were trained to transcribe meaningful English sentences processed with a noise vocoder simulation of a cochlear implant. An additional 24 subjects served as an untrained control group and transcribed the same sentences in their unprocessed form. All subjects completed pre- and post-test sessions in which they transcribed vocoded sentences to provide an assessment of training efficacy. Transfer of perceptual learning was assessed using a series of closed set, nonlinguistic tasks: subjects identified talker gender, discriminated the identity of pairs of talkers, and identified ecologically significant environmental sounds from a closed set of alternatives.

RESULTS

Although both groups of subjects showed significant pre- to post-test improvements, subjects who transcribed vocoded sentences during training performed significantly better at post-test than those in the control group. Both groups performed equally well on gender identification and talker discrimination. Subjects who received explicit training on the vocoded sentences, however, performed significantly better on environmental sound identification than the untrained subjects. Moreover, across both groups, pre-test speech performance and, to a higher degree, post-test speech performance, were significantly correlated with environmental sound identification. For both groups, environmental sounds that were characterized as having more salient temporal information were identified more often than environmental sounds that were characterized as having more salient spectral information.

CONCLUSIONS

Listeners trained to identify noise-vocoded sentences showed evidence of transfer of perceptual learning to the identification of environmental sounds. In addition, the correlation between environmental sound identification and sentence transcription indicates that subjects who were better able to use the degraded acoustic information to identify the environmental sounds were also better able to transcribe the linguistic content of novel sentences. Both trained and untrained groups performed equally well ( approximately 75% correct) on the gender-identification task, indicating that training did not have an effect on the ability to identify the gender of talkers. Although better than chance, performance on the talker discrimination task was poor overall ( approximately 55%), suggesting that either explicit training is required to discriminate talkers' voices reliably or that additional information (perhaps spectral in nature) not present in the vocoded speech is required to excel in such tasks. Taken together, the results suggest that although transfer of auditory perceptual learning with spectrally degraded speech does occur, explicit task-specific training may be necessary for tasks that cannot rely on temporal information alone.

Effects of auditory training on adult cochlear implant patients: a preliminary report

The process of learning new electrically stimulated speech patterns can be difficult for many cochlear implant users, especially congenitally deafened patients. Some implant users receive little benefit from the device, even after long-term experience. While many factors may influence individual patient outcomes, the paucity of auditory rehabilitation resources, especially for adult users, may contribute to some implant patients' poorer performance. The present study examined whether moderate auditory training, using speech stimuli, can improve the speech-recognition performance of adult cochlear implant patients. Ten cochlear implant patients with limited speech-recognition capabilities used a recently developed computer-based auditory rehabilitation tool to train at home for a period of one month or longer. Before training began, baseline speech-recognition performance was measured for each patient; baseline performance was measured for at least two weeks, until performance asymptoted. After baseline measures were complete, subjects were instructed to train themselves at home using novel monosyllable words one hour per day, five days per week. Subjects then returned to the lab every two weeks for retesting with the baseline speech materials. Preliminary results showed that there was significant improvement in all patients' speech perception performance after moderate training. While most patients did improve, the amount and time course of improvement was highly variable. Moderate training using a computer-based auditory rehabilitation tool can be an effective approach to improve cochlear implant patients' speech recognition, especially for poorer-performing implant users.

The Hybrid cochlear implant: a review

The Hybrid S or 'short-electrode' cochlear implant was developed to treat patients with a severe to profound hearing loss limited to the high frequencies. The short electrode is implanted into just the base or high-frequency region of the cochlea, with the goal of preserving residual low-frequency hearing. As a result, electric stimulation can be combined with acoustic stimulation in the same ear (and the opposite ear); this is one instance of 'acoustic plus electric' (A + E) stimulation. In this paper, we will review the latest findings from the first two stages of the clinical trial for the Hybrid concept in the United States. Generally, we will review surgical techniques, clinical trial criteria, residual hearing preservation, improvements in speech perception in quiet, and predictive factors for patient benefit. We will also discuss the significant benefit of A + E stimulation for speech perception in noise and musical measures of melody and instrument recognition, as well as valuable insights into central auditory nervous system plasticity gained from the use of a very short electrode array.

Perceptual learning for speech

Adult language users have an enormous amount of experience with speech in their native language. As a result, they have very well-developed processes for categorizing the sounds of speech that they hear. Despite this very high level of experience, recent research has shown that listeners are capable of redeveloping their speech categorization to bring it into alignment with new variation in their speech input. This reorganization of phonetic space is a type of perceptual learning, or recalibration, of speech processes. In this article, we review several recent lines of research on perceptual learning for speech.

A model of incomplete adaptation to a severely shifted frequency-to-electrode mapping by cochlear implant users

In the present study, a computational model of phoneme identification was applied to data from a previous study, wherein cochlear implant (CI) users' adaption to a severely shifted frequency allocation map was assessed regularly over 3 months of continual use. This map provided more input filters below 1 kHz, but at the expense of introducing a downwards frequency shift of up to one octave in relation to the CI subjects' clinical maps. At the end of the 3-month study period, it was unclear whether subjects' asymptotic speech recognition performance represented a complete or partial adaptation. To clarify the matter, the computational model was applied to the CI subjects' vowel identification data in order to estimate the degree of adaptation, and to predict performance levels with complete adaptation to the frequency shift. Two model parameters were used to quantify this adaptation; one representing the listener's ability to shift their internal representation of how vowels should sound, and the other representing the listener's uncertainty in consistently recalling these representations. Two of the three CI users could shift their internal representations towards the new stimulation pattern within 1 week, whereas one could not do so completely even after 3 months. Subjects' uncertainty for recalling these representations increased substantially with the frequency-shifted map. Although this uncertainty decreased after 3 months, it remained much larger than subjects' uncertainty with their clinically assigned maps. This result suggests that subjects could not completely remap their phoneme labels, stored in long-term memory, towards the frequency-shifted vowels. The model also predicted that even with complete adaptation, the frequency-shifted map would not have resulted in improved speech understanding. Hence, the model presented here can be used to assess adaptation, and the anticipated gains in speech perception expected from changing a given CI device parameter.

Efficient coding in human auditory perception

Natural sounds possess characteristic statistical regularities. Recent research suggests that mammalian auditory processing maximizes information about these regularities in its internal representation while minimizing encoding cost [Smith, E. C. and Lewicki, M. S. (2006). Nature (London) 439, 978-982]. Evidence for this "efficient coding hypothesis" comes largely from neurophysiology and theoretical modeling [Olshausen, B. A., and Field, D. (2004). Curr. Opin. Neurobiol. 14, 481-487; DeWeese, M., et al. (2003). J. Neurosci. 23, 7940-7949; Klein, D. J., et al. (2003). EURASIP J. Appl. Signal Process. 7, 659-667]. The present research provides behavioral evidence for efficient coding in human auditory perception using six-channel noise-vocoded speech, which drastically limits spectral information and degrades recognition accuracy. Two experiments compared recognition accuracy of vocoder speech created using theoretically-motivated, efficient coding filterbanks derived from the statistical regularities of speech against recognition using standard cochleotopic (logarithmic) or linear filterbanks. Recognition of the speech created using efficient encoding filterbanks was significantly more accurate than either of the other classes. These findings suggest potential applications to cochlear implant design.

Effects of envelope bandwidth on the intelligibility of sine- and noise-vocoded speech

The choice of processing parameters for vocoded signals may have an important effect on the availability of various auditory features. Experiment 1 varied envelope cutoff frequency (30 and 300 Hz), carrier type (sine and noise), and number of bands (2-5) for vocoded speech presented to normal-hearing listeners. Performance was better with a high cutoff for sine-vocoding, with no effect of cutoff for noise-vocoding. With a low cutoff, performance was better for noise-vocoding than for sine-vocoding. With a high cutoff, performance was better for sine-vocoding. Experiment 2 measured perceptibility of cues to voice pitch variations. A noise carrier combined with a high cutoff allowed intonation to be perceived to some degree but performance was best in high-cutoff sine conditions. A low cutoff led to poorest performance, regardless of carrier. Experiment 3 tested the relative contributions of co-modulation across bands and spectral density to improved performance with a sine carrier and high cutoff. Co-modulation across bands had no effect so it appears that sidebands providing a denser spectrum improved performance. These results indicate that carrier type in combination with envelope cutoff can alter the available cues in vocoded speech, factors which must be considered in interpreting results with vocoded signals.

On-line plasticity in spoken sentence comprehension: Adapting to time-compressed speech

Listeners show remarkable flexibility in processing variation in speech signal. One striking example is the ease with which they adapt to novel speech distortions such as listening to someone with a foreign accent. Behavioural studies suggest that significant improvements in comprehension occur rapidly — often within 10–20 sentences. In the present experiment, we investigate the neural changes underlying on-line adaptation to distorted speech using time-compressed speech. Listeners performed a sentence verification task on normal-speed and time-compressed sentences while their neural responses were recorded using fMRI. The results showed that rapid learning of the time-compressed speech occurred during presentation of the first block of 16 sentences and was associated with increased activation in left and right auditory association cortices and in left ventral premotor cortex. These findings suggest that the ability to adapt to a distorted speech signal may, in part, rely on mapping novel acoustic patterns onto existing articulatory motor plans, consistent with the idea that speech perception involves integrating multi-modal information including auditory and motoric cues.

Development and plasticity of intra- and intersensory information processing

The functional architecture of sensory brain regions reflects an ingenious biological solution to the competing demands of a continually changing sensory environment. While they are malleable, they have the constancy necessary to support a stable sensory percept. How does the functional organization of sensory brain regions contend with these antithetical demands? Here we describe the functional organization of auditory and multisensory (i.e., auditory-visual) information processing in three sensory brain structures: (1) a low-level unisensory cortical region, the primary auditory cortex (A1); (2) a higher-order multisensory cortical region, the anterior ectosylvian sulcus (AES); and (3) a multisensory subcortical structure, the superior colliculus (SC). We then present a body of work that characterizes the ontogenic expression of experience-dependent influences on the operations performed by the functional circuits contained within these regions. We will present data to support the hypothesis that the competing demands for plasticity and stability are addressed through a developmental transition in operational properties of functional circuits from an initially labile mode in the early stages of postnatal development to a more stable mode in the mature brain that retains the capacity for plasticity under specific experiential conditions. Finally, we discuss parallels between the central tenets of functional organization and plasticity of sensory brain structures drawn from animal studies and a growing literature on human brain plasticity and the potential applicability of these principles to the audiology clinic.

A new electrode for residual hearing preservation in cochlear implantation: first clinical results

CONCLUSION

A so far unattained high rate (100%) of residual hearing preservation in cochlear implantation for electric-acoustic stimulation could be achieved using sophisticated surgical techniques in combination with the MedEl Flex EAS electrode.

OBJECTIVES

This study aimed to gather first audiological and surgical results from the experience gained with the new MedEl Flex EAS electrode array.

PATIENTS AND METHODS

Nine patients (aged 7.62-71.32 years) with profound high frequency hearing loss were supplied with this atraumatic electrode, which was designed to preserve residual hearing despite intracochlear insertion of an electrode array. All patients were implanted by the same surgeon.

RESULTS

Hearing preservation was achieved in all patients (complete preservation 44.44%) after a mean follow-up period of 9.73 months. Mean monosyllabic test scores improved from 9% correct with the hearing aid alone to 48% with the cochlear implant and to 65% in the electric-acoustic mode.

Recognition of spectrally degraded phonemes by younger, middle-aged, and older normal-hearing listeners

The effects of spectral degradation on vowel and consonant recognition abilities were measured in young, middle-aged, and older normal-hearing (NH) listeners. Noise-band vocoding techniques were used to manipulate the number of spectral channels and frequency-to-place alignment, thereby simulating cochlear implant (CI) processing. A brief cognitive test battery was also administered. The performance of younger NH listeners exceeded that of the middle-aged and older listeners, when stimuli were severely distorted (spectrally shifted); the older listeners performed only slightly worse than the middle-aged listeners. Significant intragroup variability was present in the middle-aged and older groups. A hierarchical multiple-regression analysis including data from all three age groups suggested that age was the primary factor related to shifted vowel recognition performance, but verbal memory abilities also contributed significantly to performance. A second regression analysis (within the middle-aged and older groups alone) revealed that verbal memory and speed of processing abilities were better predictors of performance than age alone. The overall results from the current investigation suggested that both chronological age and cognitive capacities contributed to the ability to recognize spectrally degraded phonemes. Such findings have important implications for the counseling and rehabilitation of adult CI recipients.

Identification of frequency-shifted vowels

Within certain limits, speech intelligibility is preserved with upward or downward scaling of the spectral envelope. To study these limits and assess their interaction with fundamental frequency (F0), vowels in /hVd/ syllables were processed using the STRAIGHT vocoder and presented to listeners for identification. Identification accuracy showed a gradual decline when the spectral envelope was scaled up or down in vowels spoken by men, women, and children. Upward spectral envelope shifts led to poorer identification of children's vowels compared to adults, while downward shifts had a greater impact on men's vowels compared to women and children. Coordinated shifts (F0 and spectral envelope shifted in the same direction) generally produced higher accuracy than conditions with F0 and spectral envelope shifted in opposite directions. Vowel identification was poorest in conditions with very high F0, consistent with suggestions from the literature that sparse sampling of the spectral envelope may be a factor in vowel identification. However, the gradual decline in accuracy as a function of both upward and downward spectral envelope shifts and the interaction between spectral envelope shifts and F0 suggests the additional operation of perceptual mechanisms sensitive to the statistical covariation of F0 and formant frequencies in natural speech.

Lexical tone recognition with spectrally mismatched envelopes

It has been shown that frequency-place mismatch has detrimental effects on English speech recognition. The present study investigated the effects of mismatched spectral distribution of envelopes on Mandarin Chinese tone recognition using a noise-excited vocoder. In Experiment 1, speech samples were processed to simulate a cochlear implant with various insertion depths. The carrier bands were shifted basally relative to the analysis bands by 1-7 mm in the cochlea. Nine normal-hearing Mandarin Chinese listeners participated in this experiment. Basal shift of the carriers only slightly affected tone recognition. The resistance of tone recognition to spectral shift can be attributed to the overall amplitude contour cues that are independent from spectral manipulations. Experiment 2 examined the effects of frequency compression, where widened analysis bands by 2, 6, and 10 mm were compressively allocated to narrower carrier bands. Five of the 9 subjects participated in Experiment 2. It appears that the expanded frequency information especially on the low frequency end can compensate for the distortion from frequency compression. Thus, spectral shift might not pose a severe problem for tone recognition, and allocation of wider frequency range to include more low frequency information might be beneficial for tone recognition.

Comparison of word-, sentence-, and phoneme-based training strategies in improving the perception of spectrally distorted speech

PURPOSE

To compare the effectiveness of 3 self-administered strategies for auditory training that might improve speech perception by adult users of cochlear implants. The strategies are based, respectively, on discriminating isolated words, words in sentences, and phonemes in nonsense syllables.

METHOD

Participants were 18 normal-hearing adults who listened to speech processed by a noise-excited vocoder to simulate the information provided by a cochlear implant. They were assigned randomly to word-, sentence-, or phoneme-based training and underwent 9 training sessions (20 min each) on separate days over a 2- to 3-week period. The effectiveness of training was assessed as the improvement in accuracy of discriminating vowels and consonants, as well as identifying words in sentences, relative to participants' best performance in repeated tests prior to training.

RESULTS

Word- and sentence-based training led to significant improvements in the ability to identify words in sentences that were significantly larger than the improvements produced by phoneme-based training. There were no significant differences between the effectiveness of word- and sentence-based training. No significant improvements in consonant or vowel discrimination were found for the sentence- or phoneme-based training groups, but some improvements were found for the word-based training group.

CONCLUSION

The word- and sentence-based training strategies were more effective than the phoneme-based strategy at improving the perception of spectrally distorted speech.

Effects of upper-frequency boundary and spectral warping on speech intelligibility in electrical stimulation

Speech understanding was tested for seven listeners using 12-electrode Med-El cochlear implants (CIs) and six normal-hearing listeners using a CI simulation. Eighteen different types of processing were evaluated, which varied the frequency-to-tonotopic place mapping and the upper boundary of the frequency and stimulation range. Spectrally unwarped and warped conditions were included. Unlike previous studies on this topic, the lower boundary of the frequency and stimulation range was fixed while the upper boundary was varied. For the unwarped conditions, only eight to ten channels were needed in both quiet and noise to achieve no significant degradation in speech understanding compared to the normal 12-electrode speech processing. The unwarped conditions were often the best conditions for understanding speech; however, small changes in frequency-to-place mapping (<0.77 octaves for the most basal electrode) yielded no significant degradation in performance from the nearest unwarped condition. A second experiment measured the effect of feedback training for both the unwarped and warped conditions. Improvements were found for the unwarped and frequency-expanded conditions, but not for the compressed condition. These results have implications for new CI processing strategies, such as the inclusion of spectral localization cues.

Cochlear implant speech processor frequency allocations may influence pitch perception

OBJECTIVE

To investigate the effects of assigning cochlear implant speech processor frequencies normally associated with more apical cochlear locations to the shallow insertion depths of the Iowa/Nucleus Hybrid electrode.

STUDY DESIGN

Subjects using the Hybrid implant for more than 1 year were tested on speech recognition with Consonant-Nucleus-Consonant words and consonant stimuli. Pitch sensations of individual electrodes were also measured electrically through the implant and acoustically in the contralateral ear.

SETTING

Tertiary care center.

RESULTS

Most subjects showed large improvements in speech recognition within 12 months after implantation. Furthermore, after longer periods of 24-plus months, some individuals were able to achieve high levels of consonant discrimination with electric-only processing comparable to long-electrode patients with deeper electrode insertions. Pitch perceptions obtained from individual electrodes in these subjects were closer to the frequency map assigned an electrode than the place-frequency predicted from cochlear location.

CONCLUSION

These results suggest that over time, pitch sensations may be determined more by the implant map than by cochlear location. In other words, the brain may adapt to spectral mismatches by remapping pitch. Furthermore, patients can perform well with shifted frequency allocations for speech recognition. The successful application of shifted frequency allocations also supports the idea of shallower insertions and greater preservation of residual hearing for all cochlear implants, regardless of the patient's frequency range of usable residual hearing.

Perceptual learning and auditory training in cochlear implant recipients

Learning electrically stimulated speech patterns can be a new and difficult experience for cochlear implant (CI) recipients. Recent studies have shown that most implant recipients at least partially adapt to these new patterns via passive, daily-listening experiences. Gradually introducing a speech processor parameter (eg, the degree of spectral mismatch) may provide for more complete and less stressful adaptation. Although the implant device restores hearing sensation and the continued use of the implant provides some degree of adaptation, active auditory rehabilitation may be necessary to maximize the benefit of implantation for CI recipients. Currently, there are scant resources for auditory rehabilitation for adult, postlingually deafened CI recipients. We recently developed a computer-assisted speech-training program to provide the means to conduct auditory rehabilitation at home. The training software targets important acoustic contrasts among speech stimuli, provides auditory and visual feedback, and incorporates progressive training techniques, thereby maintaining recipients' interest during the auditory training exercises. Our recent studies demonstrate the effectiveness of targeted auditory training in improving CI recipients' speech and music perception. Provided with an inexpensive and effective auditory training program, CI recipients may find the motivation and momentum to get the most from the implant device.

Relationship between perception of spectral ripple and speech recognition in cochlear implant and vocoder listeners

Spectral resolution has been reported to be closely related to vowel and consonant recognition in cochlear implant (CI) listeners. One measure of spectral resolution is spectral modulation threshold (SMT), which is defined as the smallest detectable spectral contrast in the spectral ripple stimulus. SMT may be determined by the activation pattern associated with electrical stimulation. In the present study, broad activation patterns were simulated using a multi-band vocoder to determine if similar impairments in speech understanding scores could be produced in normal-hearing listeners. Tokens were first decomposed into 15 logarithmically spaced bands and then re-synthesized by multiplying the envelope of each band by matched filtered noise. Various amounts of current spread were simulated by adjusting the drop-off of the noise spectrum away from the peak (40-5 dBoctave). The average SMT (0.25 and 0.5 cyclesoctave) increased from 6.3 to 22.5 dB, while average vowel identification scores dropped from 86% to 19% and consonant identification scores dropped from 93% to 59%. In each condition, the impairments in speech understanding were generally similar to those found in CI listeners with similar SMTs, suggesting that variability in spread of neural activation largely accounts for the variability in speech perception of CI listeners.

Melodic contour identification by cochlear implant listeners

OBJECTIVE

While the cochlear implant provides many deaf patients with good speech understanding in quiet, music perception and appreciation with the cochlear implant remains a major challenge for most cochlear implant users. The present study investigated whether a closed-set melodic contour identification (MCI) task could be used to quantify cochlear implant users' ability to recognize musical melodies and whether MCI performance could be improved with moderate auditory training. The present study also compared MCI performance with familiar melody identification (FMI) performance, with and without MCI training.

METHODS

For the MCI task, test stimuli were melodic contours composed of 5 notes of equal duration whose frequencies corresponded to musical intervals. The interval between successive notes in each contour was varied between 1 and 5 semitones; the "root note" of the contours was also varied (A3, A4, and A5). Nine distinct musical patterns were generated for each interval and root note condition, resulting in a total of 135 musical contours. The identification of these melodic contours was measured in 11 cochlear implant users. FMI was also evaluated in the same subjects; recognition of 12 familiar melodies was tested with and without rhythm cues. MCI was also trained in 6 subjects, using custom software and melodic contours presented in a different frequency range from that used for testing.

RESULTS

Results showed that MCI recognition performance was highly variable among cochlear implant users, ranging from 14% to 91% correct. For most subjects, MCI performance improved as the number of semitones between successive notes was increased; performance was slightly lower for the A3 root note condition. Mean FMI performance was 58% correct when rhythm cues were preserved and 29% correct when rhythm cues were removed. Statistical analyses revealed no significant correlation between MCI performance and FMI performance (with or without rhythmic cues). However, MCI performance was significantly correlated with vowel recognition performance; FMI performance was not correlated with cochlear implant subjects' phoneme recognition performance. Preliminary results also showed that the MCI training improved all subjects' MCI performance; the improved MCI performance also generalized to improved FMI performance.

CONCLUSIONS

Preliminary data indicate that the closed-set MCI task is a viable approach toward quantifying an important component of cochlear implant users' music perception. The improvement in MCI performance and generalization to FMI performance with training suggests that MCI training may be useful for improving cochlear implant users' music perception and appreciation; such training may be necessary to properly evaluate patient performance, as acute measures may underestimate the amount of musical information transmitted by the cochlear implant device and received by cochlear implant listeners.

Effect of training rate on recognition of spectrally shifted speech

OBJECTIVE

Previous studies have shown that the protocol used for auditory training may significantly affect the outcome of training. However, it is unclear how often training should be performed to maximize its benefit. The present study investigated how the frequency of training contributed to normal-hearing listeners' adaptation to spectrally shifted speech.

METHODS

Eighteen normal-hearing listeners were trained with spectrally shifted and compressed speech via an 8-channel acoustic simulation of cochlear implant speech processing. Five short training sessions (1 hr per session) were completed by each subject; subjects were trained at one of three training rates: five sessions per week, three sessions per week, or one session per week. Subjects were trained to identify medial vowels presented in a cVc format; depending on the level of difficulty, the number of response choices was increased and/or the acoustic differences between vowels were reduced. Vowel and consonant recognition was measured before and after training as well as at regular intervals during the training period. Sentence recognition was measured before and after training only.

RESULTS

Results showed that pretraining vowel recognition scores were poor (14.0% correct, on average) for all subjects, due to the severe spectral shift. After five sessions of targeted vowel contrast training, there was a significant improvement of shifted vowel recognition for most subjects. The mean improvement was comparable (approximately 15 percentage points) across the three training rate conditions, despite significant intersubject variability in pre- and pretraining baseline performance. There was no significant difference in training outcomes among the three training rates. Spectrally shifted consonant and sentence recognition also improved by approximately 20 percentage points after training, even though consonants and sentences were not explicitly trained. Similar to vowel recognition, there was no significant difference in training outcomes among the three training rates for shifted consonant and sentence recognition.

CONCLUSIONS

The results demonstrated that the training rate had little effect on normal-hearing listeners' adaptation to spectrally shifted speech, at least for the training periods (ranging from 1 to 5 wk) used in the present study. The outcome of auditory training may depend more strongly on the amount of training (i.e., total number of training sessions) rather than the frequency of training (i.e., daily or once per week). Although more frequent training may accelerate listeners' adaptation to spectrally shifted speech, there may be significant benefits from training as little as one session per week. The results of the present study suggest that appropriate training schedules can be developed to optimize the effectiveness, efficiency, and effort associated with hearing-impaired patients' auditory rehabilitation.

Speech recognition as a function of high-pass filter cutoff frequency for people with and without low-frequency cochlear dead regions

Regions in the cochlea with no (or very few) functioning inner hair cells and/or neurons are called "dead regions" (DRs). The recognition of high-pass filtered nonsense syllables was measured as a function of filter cutoff frequency for hearing-impaired people with and without low-frequency (apical) cochlear DRs. The diagnosis of any DR was made using the TEN(HL) test, and psychophysical tuning curves were used to define the edge frequency (f(e)) more precisely. Stimuli were amplified differently for each ear, using the "Cambridge formula." For subjects with low-frequency hearing loss but without DRs, scores were high (about 78%) for low cutoff frequencies, remained approximately constant for cutoff frequencies up to 862 Hz, and then worsened with increasing cutoff frequency. For subjects with low-frequency DRs, performance was typically poor for the lowest cutoff frequency (100 Hz), improved as the cutoff frequency was increased to about 0.57f(e), and worsened with further increases. These results indicate that people with low-frequency DRs are able to make effective use of frequency components that fall in the range 0.57f(e) to f(e), but that frequency components below 0.57f(e) have deleterious effects. The results have implications for the fitting of hearing aids to people with low-frequency DRs.

Combined Effects of Frequency Compression-Expansion and Shift on Speech Recognition

OBJECTIVE

To explore combined acute effects of frequency shift and compression-expansion on speech recognition, using noiseband vocoder processing.

DESIGN

Recognition of vowels and consonants, processed with a noiseband vocoder, was measured with five normal-hearing subjects, between the ages of 27 and 35 yr. The speech signal was filtered into 8 or 16 analysis bands and the envelopes were extracted from each band. The carrier noise bands were modulated by the envelopes and resynthesized to produce the processed speech. In the baseline matched condition, the frequency ranges of the corresponding analysis and carrier bands were the same. In the shift only condition, the frequency ranges of the carrier bands were shifted up or down relative to the analysis bands. In the compression and expansion only conditions, the analysis band range was made larger or smaller, respectively, than the carrier band range. By applying the shift to carrier bands and compression or expansion to analysis bands simultaneously, the combined effects of the two spectral distortions on speech recognition were explored.

RESULTS

When the spectral distortions of compression-expansion or shift were applied separately, the performance was reduced from the baseline matched condition. However, when the two spectral degradations were applied simultaneously, a compensatory effect was observed; the reduction in performance was smaller for some combinations compared to the reduction observed for each distortion individually.

CONCLUSIONS

The results of the present study are consistent with previous vocoder studies with normal-hearing subjects that showed a negative effect of spectral mismatch between analysis and carrier bands on speech recognition. The present results further show that matching the frequency ranges of 1 to 2 kHz, which contain important speech information, can be more beneficial for speech recognition than matching the overall frequency ranges, in certain conditions.

Changes in pitch with a cochlear implant over time

In the normal auditory system, the perceived pitch of a tone is closely linked to the cochlear place of vibration. It has generally been assumed that high-rate electrical stimulation by a cochlear implant electrode also evokes a pitch sensation corresponding to the electrode's cochlear place ("place" code) and stimulation rate ("temporal" code). However, other factors may affect electric pitch sensation, such as a substantial loss of nearby nerve fibers or even higher-level perceptual changes due to experience. The goals of this study were to measure electric pitch sensations in hybrid (short-electrode) cochlear implant patients and to examine which factors might contribute to the perceived pitch. To look at effects of experience, electric pitch sensations were compared with acoustic tone references presented to the non-implanted ear at various stages of implant use, ranging from hookup to 5 years. Here, we show that electric pitch perception often shifts in frequency, sometimes by as much as two octaves, during the first few years of implant use. Additional pitch measurements in more recently implanted patients at shorter time intervals up to 1 year of implant use suggest two likely contributions to these observed pitch shifts: intersession variability (up to one octave) and slow, systematic changes over time. We also found that the early pitch sensations for a constant electrode location can vary greatly across subjects and that these variations are strongly correlated with speech reception performance. Specifically, patients with an early low-pitch sensation tend to perform poorly with the implant compared to those with an early high-pitch sensation, which may be linked to less nerve survival in the basal end of the cochlea in the low-pitch patients. In contrast, late pitch sensations show no correlation with speech perception. These results together suggest that early pitch sensations may more closely reflect peripheral innervation patterns, while later pitch sensations may reflect higher-level, experience-dependent changes. These pitch shifts over time not only raise questions for strict place-based theories of pitch perception, but also imply that experience may have a greater influence on cochlear implant perception than previously thought.

Cochlear Implant Channel Separation and Its Influence on Speech Perception – Implications for a New Electrode Design

There are a variety of factors which can influence cochlear implantation outcome. Channel interaction is one of the variables responsible for audiological performance deterioration in multichannel implants. Electrode design is--among others--one way to decrease the incidence of channel interaction. At present, electrodes differ in overall length, diameter, contact design and distribution, but none of the electrodes available have a distinct variability in the amount of space between contacts across the length of the electrode. The aim of this study was to investigate whether a new electrode design featuring larger contact spacing in the apical part of deeply inserted electrodes would lead to an increase in speech perception. Eighteen postlingually deafened patients fitted with MedEl Combi 40+ or MedEl Pulsar cochlear implants using the MedEl implementation of continuous interleaved sampling participated in this study. Patients were tested in 6 conditions, in which the channel spacing and distribution of electrode contacts in each patient were artificially varied by activating or deactivating different channels. Performance was tested immediately after each change in setup with a monosyllable and sentence test (Hochmaier, Schultz and Moser). Our results showed that the condition with the highest distance between contacts in the apical part (up to 6.4 mm instead of 2.4 mm) is the most effective for the matched map condition: the results improved statistically significantly for the sentence test from 72% in the standard 12-channel condition to 83.2% and from 40.8 to 50% for the monosyllable test. Based on these findings, we present a new electrode design which can help achieve further increases in speech perception with cochlear implants.

Effectiveness of computer-based auditory training in improving the perception of noise-vocoded speech

Five experiments were designed to evaluate the effectiveness of "high-variability" lexical training in improving the ability of normal-hearing subjects to perceive noise-vocoded speech that had been spectrally shifted to simulate tonotopic misalignment. Two approaches to training were implemented. One training approach required subjects to recognize isolated words, while the other training approach required subjects to recognize words in sentences. Both approaches to training improved the ability to identify words in sentences. Improvements following a single session (lasting 1-2 h) of auditory training ranged between 7 and 12 %pts and were significantly larger than improvements following a visual control task that was matched with the auditory training task in terms of the response demands. An additional three sessions of word- and sentence-based training led to further improvements, with the average overall improvement ranging from 13 to 18% pts. When a tonotopic misalignment of 3 mm rather than 6 mm was simulated, training with several talkers led to greater generalization to new talkers than training with a single talker. The results confirm that computer-based lexical training can help overcome the effects of spectral distortions in speech, and they suggest that training materials are most effective when several talkers are included.

Computer-Assisted Speech Training for Cochlear Implant Patients: Feasibility, Outcomes, and Future Directions

Learning electrically stimulated speech patterns can be a new and difficult experience for cochlear implant patients. Cochlear implantation alone may not fully meet the needs of many patients, and additional auditory rehabilitation may be necessary to maximize the benefits of the implant device. A recently developed computer-assisted speech-training program provides cochlear implant patients with the means to conduct auditory rehabilitation at home. The training software targets important acoustic contrasts between speech stimuli and provides auditory and visual feedback as well as progressive training, thereby maintaining patients' interest in the auditory training exercises. Recent scientific studies have demonstrated the effectiveness of such specialized auditory training programs in improving cochlear implant patients' speech recognition performance. Provided with an inexpensive and accessible auditory training program, cochlear implant patients may find the motivation and momentum to get the most from the implant device.

Continuous improvement in Mandarin lexical tone perception as the number of channels increased: a simulation study of cochlear implant

CONCLUSION

With reference to English phoneme recognition, where performance usually does not improve after six or eight channels in cochlear implants (CIs), increasing total channel numbers continuously improved perception of Mandarin tones.

OBJECTIVE

To test our hypothesis that current CI strategies might be modified to improve Mandarin lexical tonal perception.

MATERIALS AND METHODS

Lexical tonal perception tests using 48 monosyllables in Mandarin Chinese were conducted in 32 native Mandarin speakers with normal hearing. The performance of tonal perception was compared among the controlled factors, which were total channel number, number of channels allocated to the F0 spectrum, and whether there were spectral shifts in the electrode configuration. The experimental condition that preserves fine structure was used as a comparison.

RESULTS

The signal processing strategy using 16 channels--which is technically possible with current CI devices--produced better tonal perception than those using 12 or 8 channels. Increasing the number of fundamental channels did not improve tonal perception, and spectral shifts did not change tonal perception. An experimental condition (FiC12) that preserves the fine structure produced significantly better overall scores for tone perception than other experimental conditions with envelope strategies.

Balance sensory organization in children with profound hearing loss and cochlear implants

OBJECTIVES

(1) To determine the feasibility of the use of a modified postural control test under altered sensory conditions in children over 8 years of age, and (2) to assess how deaf children use sensory information for postural control when they have normal or abnormal vestibular responses, and if hearing input from a unilateral cochlear implant, changes their postural behavior.

PATIENTS

We selected 36 children, 8 to 11 years of age, with congenital or early-acquired profound sensorineural hearing loss, 13 of them with unilateral cochlear implantation and 22 normal-hearing children.

METHODS

The Postural Control (PC) test consists of a force platform with 2 stimulation paradigm conditions: (1) standing on the platform with opened eyes; (2) standing on foam placed on the force platform with closed eyes. Implanted children were tested with the implant turn on and turn off in this condition, in order to evaluate eventual change in the postural control parameters when they have hearing habilitation. The body center of pressure distribution area (COP) and the body sway velocity (SV) were the parameter to evaluate the postural control.

RESULTS

Deaf children were classified into two groups according with the vestibular responses: group A (n=28) Children with normal vestibular rotary responses; group B (n=8) children with hypoactive responses. Children in group A had diagnoses of syndromic and non-syndromic hereditary deafness, and children in group B had inner ear malformations, post-meningitis deafness, and one child had non-syndromic hereditary deafness with hypoactive vestibular response. In condition 1, when vestibular, somatosensory and visual information were enabled, the COP and SV values did not show any statistically significant differences between groups A, B and control. In condition 2, when visual information was removed and the somatosensory input strongly modified by standing on the foam, group B showed significant higher COP and SV values than groups A and control (p<0.05). In addition, the scalograms by wavelets of children in group B had higher amplitudes increasing the sway frequencies contents up to 3 Hz, not allowing them to maintain the up right stance in similar stimulation than in condition. Implanted children of the group A and B with the implant turn on, in the condition 2, did not show any significant difference in the SV, comparing when they had the implanted turn off. Group A p=0.395 and group B p=0.465 (Wilcoxon ranked test).

CONCLUSION

These findings allow us to confirm that this postural test can be performed in children over 8 years old. Also our results suggest that deaf children with associated hypoactive vestibular responses included in our study, despite the etiology of the deafness, primarily use visual and somatosensory information to maintain their postural control. Hearing habilitation with a unilateral cochlear implant has no effect on the observed sensory organization strategy.

An electric frequency-to-place map for a cochlear implant patient with hearing in the nonimplanted ear

The aim of this study was to relate the pitch of high-rate electrical stimulation delivered to individual cochlear implant electrodes to electrode insertion depth and insertion angle. The patient (CH1) was able to provide pitch matches between electric and acoustic stimulation because he had auditory thresholds in his nonimplanted ear ranging between 30 and 60 dB HL over the range, 250 Hz to 8 kHz. Electrode depth and insertion angle were measured from high-resolution computed tomography (CT) scans of the patient's temporal bones. The scans were used to create a 3D image volume reconstruction of the cochlea, which allowed visualization of electrode position within the scala. The method of limits was used to establish pitch matches between acoustic pure tones and electric stimulation (a 1,652-pps, unmodulated, pulse train). The pitch matching data demonstrated that, for insertion angles of greater than 450 degrees or greater than approximately 20 mm insertion depth, pitch saturated at approximately 420 Hz. From 20 to 15 mm insertion depth pitch estimates were about one-half octave lower than the Greenwood function. From 13 to 3 mm insertion depth the pitch estimates were approximately one octave lower than the Greenwood function. The pitch match for an electrode only 3.4 mm into the cochlea was 3,447 Hz. These data are consistent with other reports, e.g., Boëx et al. (2006), of a frequency-to-place map for the electrically stimulated cochlea in which perceived pitches for stimulation on individual electrodes are significantly lower than those predicted by the Greenwood function for stimulation at the level of the hair cell.

Effects of stimulus variability and adult aging on adaptation to time-compressed speech

With as few as 10-20 sentences of exposure, listeners are able to adapt to speech that is highly distorted compared to that which is encountered in everyday conversation. The current study examines the extent to which adaptation to time-compressed speech can be impeded by disrupting the continuity of the exposure sentences, and whether this differs between young and older adult listeners when they are equated for starting accuracy. In separate sessions conducted one week apart, the degree of adaptation was assessed in four exposure conditions, all of which involved exposure to the same number of time-compressed sentences. A continuous exposure condition involved presentation of the time-compressed sentences without interruption. Two alternation conditions alternated time-compressed speech and uncompressed speech by single sentences or groups of four sentences. A fourth condition presented sentences that were separated by a period of silence but no uncompressed speech. For all conditions, neither young nor older adults' overall level of learning was influenced by disruptions to the exposure sentences. In addition, participants' performance showed reliable improvement across the first and subsequent sessions. These results support robust learning mechanisms in speech perception that remain functional throughout the lifespan.

Frequency map for the human cochlear spiral ganglion: implications for cochlear implants

The goals of this study were to derive a frequency-position function for the human cochlear spiral ganglion (SG) to correlate represented frequency along the organ of Corti (OC) to location along the SG, to determine the range of individual variability, and to calculate an "average" frequency map (based on the trajectories of the dendrites of the SG cells). For both OC and SG frequency maps, a potentially important limitation is that accurate estimates of cochlear place frequency based upon the Greenwood function require knowledge of the total OC or SG length, which cannot be determined in most temporal bone and imaging studies. Therefore, an additional goal of this study was to evaluate a simple metric, basal coil diameter that might be utilized to estimate OC and SG length. Cadaver cochleae (n = 9) were fixed <24 h postmortem, stained with osmium tetroxide, microdissected, decalcified briefly, embedded in epoxy resin, and examined in surface preparations. In digital images, the OC and SG were measured, and the radial nerve fiber trajectories were traced to define a series of frequency-matched coordinates along the two structures. Images of the cochlear turns were reconstructed and measurements of basal turn diameter were made and correlated with OC and SG measurements. The data obtained provide a mathematical function for relating represented frequency along the OC to that of the SG. Results showed that whereas the distance along the OC that corresponds to a critical bandwidth is assumed to be constant throughout the cochlea, estimated critical band distance in the SG varies significantly along the spiral. Additional findings suggest that measurements of basal coil diameter in preoperative images may allow prediction of OC/SG length and estimation of the insertion depth required to reach specific angles of rotation and frequencies. Results also indicate that OC and SG percentage length expressed as a function of rotation angle from the round window is fairly constant across subjects. The implications of these findings for the design and surgical insertion of cochlear implants are discussed.

On the structure of phoneme categories in listeners with cochlear implants

PURPOSE

To describe cochlear implant users' phoneme labeling, discrimination, and prototypes for a vowel and a sibilant contrast, and to assess the effects of 1 year's experience with prosthetic hearing.

METHOD

Based on naturally produced clear examples of "boot," "beet," "said," and "shed" by 1 male and 1 female speaker, continua with 13 stimuli were synthesized for each contrast. Seven hearing controls labeled those stimuli and assigned them goodness ratings, as did 7 implant users at 1-month postimplant. One year later, these measures were repeated, and within category discrimination, d', was assessed.

RESULTS

Compared with controls, implant users' vowel and sibilant labeling slopes were substantially shallower but improved over 1 year of prosthesis use. Their sensitivity to phonetic differences within phoneme categories was about half that of controls. The slopes of their goodness rating functions were shallower and did not improve. Their prototypes for the sibilant contrast (but not the vowels) were closer to one another and did not improve by moving apart.

CONCLUSIONS

Implant users' phoneme labeling and within-category perceptual structure were anomalous at 1-month postimplant. After 1 year of prosthesis use, phoneme labeling categories had sharpened but within category discrimination was well below that of hearing controls.

Using evoked potentials to match interaural electrode pairs with bilateral cochlear implants

Bilateral cochlear implantation seeks to restore the advantages of binaural hearing to the profoundly deaf by providing binaural cues normally important for accurate sound localization and speech reception in noise. Psychophysical observations suggest that a key issue for the implementation of a successful binaural prosthesis is the ability to match the cochlear positions of stimulation channels in each ear. We used a cat model of bilateral cochlear implants with eight-electrode arrays implanted in each cochlea to develop and test a noninvasive method based on evoked potentials for matching interaural electrodes. The arrays allowed the cochlear location of stimulation to be independently varied in each ear. The binaural interaction component (BIC) of the electrically evoked auditory brainstem response (EABR) was used as an assay of binaural processing. BIC amplitude peaked for interaural electrode pairs at the same relative cochlear position and dropped with increasing cochlear separation in either direction. To test the hypothesis that BIC amplitude peaks when electrodes from the two sides activate maximally overlapping neural populations, we measured multiunit neural activity along the tonotopic gradient of the inferior colliculus (IC) with 16-channel recording probes and determined the spatial pattern of IC activation for each stimulating electrode. We found that the interaural electrode pairings that produced the best aligned IC activation patterns were also those that yielded maximum BIC amplitude. These results suggest that EABR measurements may provide a method for assigning frequency-channel mappings in bilateral implant recipients, such as pediatric patients, for which psychophysical measures of pitch ranking or binaural fusion are unavailable.

Implications of deep electrode insertion on cochlear implant fitting

Using long Med-El Combi40+ electrode arrays, it is now possible to cover the whole range of the cochlea, up to about two turns. Such insertion depths have received little attention. To evaluate the contribution of deeply inserted electrodes, five Med-El cochlear implant users were tested on vowel and consonant identification tests with fittings with first one, two, and up to five apical electrodes being deactivated. In addition, subjects performed pitch-ranking experiments, using loudness-balanced stimuli, to identify electrodes creating pitch confusions. Radiographs were taken to measure each electrode insertion depth. All subjects used each modified fitting for two periods of about 3 weeks. During the experiment, the same stimulation rate and frequency range were maintained across all the fittings used for each individual subject. After each trial period the subject had to perform three consonant and three vowel identification tests. All subjects showed deep electrode insertions ranging from 605 degrees to 720 degrees. The two subjects with the deepest electrode insertions showed significantly increased vowel- and consonant-identification performances with fittings with the two or three most apical electrodes deactivated compared to their standard fitting with all available electrodes activated. The other three subjects did not show significant improvements in performance when one or two of their most apical electrodes were deactivated. Four out of five subjects preferred to continue use of a fitting with one or more apical electrodes deactivated. The two subjects with the deepest insertions also showed pitch confusions between their most apical electrodes. Two possible reasons for these results are discussed. One is to reduce neural interactions related to electrodes producing pitch confusions. Another is to improve the alignment of the frequency components of sounds coded by the electrical signals delivered to each electrode to the overall pitch of the auditory perception produced by the electrical stimulation of auditory nerve fibers.

Estimation of Vowel Recognition With Cochlear Implant Simulations

Because there are many parameters in the cochlear implant (CI) device that can be optimized for individual patients, it is important to estimate a parameter's effect before patient evaluation. In this paper, Mel-frequency cepstrum coefficients (MFCCs) were used to estimate the acoustic vowel space for vowel stimuli processed by the CI simulations. The acoustic space was then compared to vowel recognition performance by normal-hearing subjects listening to the same processed speech. Five CI speech processor parameters were simulated to produce different degree of spectral resolution, spectral smearing, spectral warping, spectral shifting, and amplitude distortion. The acoustic vowel space was highly correlated with normal hearing subjects' vowel recognition performance for parameters that affected the spectral channels and spectral smearing. However, the acoustic vowel space was not significantly correlated with perceptual performance for parameters that affected the degree of spectral warping, spectral shifting, and amplitude distortion. In particular, while spectral warping and shifting did not significantly reshape the acoustic space, vowel recognition performance was significantly affected by these parameters. The results from the acoustic analysis suggest that the CI device can preserve phonetic distinctions under conditions of spectral warping and shifting. Auditory training may help CI patients better perceive these speech cues transmitted by their speech processors.

Vowel recognition via cochlear implants and noise vocoders: effects of formant movement and duration

Previous work has demonstrated that normal-hearing individuals use fine-grained phonetic variation, such as formant movement and duration, when recognizing English vowels. The present study investigated whether these cues are used by adult postlingually deafened cochlear implant users, and normal-hearing individuals listening to noise-vocoder simulations of cochlear implant processing. In Experiment 1, subjects gave forced-choice identification judgments for recordings of vowels that were signal processed to remove formant movement and/or equate vowel duration. In Experiment 2, a goodness-optimization procedure was used to create perceptual vowel space maps (i.e., best exemplars within a vowel quadrilateral) that included F1, F2, formant movement, and duration. The results demonstrated that both cochlear implant users and normal-hearing individuals use formant movement and duration cues when recognizing English vowels. Moreover, both listener groups used these cues to the same extent, suggesting that postlingually deafened cochlear implant users have category representations for vowels that are similar to those of normal-hearing individuals.

Perceptual adaptation by normally hearing listeners to a simulated "hole" in hearing

Simulations of cochlear implants have demonstrated that the deleterious effects of a frequency misalignment between analysis bands and characteristic frequencies at basally shifted simulated electrode locations are significantly reduced with training. However, a distortion of frequency-to-place mapping may also arise due to a region of dysfunctional neurons that creates a "hole" in the tonotopic representation. This study simulated a 10 mm hole in the mid-frequency region. Noise-band processors were created with six output bands (three apical and three basal to the hole). The spectral information that would have been represented in the hole was either dropped or reassigned to bands on either side. Such reassignment preserves information but warps the place code, which may in itself impair performance. Normally hearing subjects received three hours of training in two reassignment conditions. Speech recognition improved considerably with training. Scores were much lower in a baseline (untrained) condition where information from the hole region was dropped. A second group of subjects trained in this dropped condition did show some improvement; however, scores after training were significantly lower than in the reassignment conditions. These results are consistent with the view that speech processors should present the most informative frequency range irrespective of frequency misalignment.

Amplitude modulation of turbulence noise by voicing in fricatives

The two principal sources of sound in speech, voicing and frication, occur simultaneously in voiced fricatives as well as at the vowel-fricative boundary in phonologically voiceless fricatives. Instead of simply overlapping, the two sources interact. This paper is an acoustic study of one such interaction effect: the amplitude modulation of the frication component when voicing is present. Corpora of sustained and fluent-speech English fricatives were recorded and analyzed using a signal-processing technique designed to extract estimates of modulation depth. Results reveal a pattern, consistent across speaking style, speaker, and place of articulation, for modulation at fo to rise at low voicing strengths and subsequently saturate. Voicing strength needed to produce saturation varied 60-66 dB across subjects and experimental conditions. Modulation depths at saturation varied little across speakers but significantly for place of articulation (with [z] showing particularly strong modulation) clustering at approximately 0.4-0.5 (a 40%-50% fluctuation above and below unmodulated amplitude); spectral analysis of modulating signals revealed weak but detectable modulation at the second and third harmonics (i.e., 2fo and 3fo).

Perceptual adaptation to spectrally shifted vowels: training with nonlexical labels

Although normal-hearing (NH) and cochlear implant (CI) listeners are able to adapt to spectrally shifted speech to some degree, auditory training has been shown to provide more complete and/or accelerated adaptation. However, it is unclear whether listeners use auditory and visual feedback to improve discrimination of speech stimuli, or to learn the identity of speech stimuli. The present study investigated the effects of training with lexical and nonlexical labels on NH listeners' perceptual adaptation to spectrally degraded and spectrally shifted vowels. An eight-channel sine wave vocoder was used to simulate CI speech processing. Two degrees of spectral shift (moderate and severe shift) were studied with three training paradigms, including training with lexical labels (i.e., "hayed," "had," "who'd," etc.), training with nonlexical labels (i.e., randomly assigned letters "f," "b," "g," etc.), and repeated testing with lexical labels (i.e., "test-only" paradigm without feedback). All training and testing was conducted over 5 consecutive days, with two to four training exercises per day. Results showed that with the test-only paradigm, lexically labeled vowel recognition significantly improved for moderately shifted vowels; however, there was no significant improvement for severely shifted vowels. Training with nonlexical labels significantly improved the recognition of nonlexically labeled vowels for both shift conditions; however, this improvement failed to generalize to lexically labeled vowel recognition with severely shifted vowels. Training with lexical labels significantly improved lexically labeled vowel recognition with severely shifted vowels. These results suggest that storage and retrieval of speech patterns in the central nervous system is somewhat robust to tonotopic distortion and spectral degradation. Although training with nonlexical labels may improve discrimination of spectrally distorted peripheral patterns, lexically meaningful feedback is needed to identify these peripheral patterns. The results also suggest that training with lexically meaningful feedback may be beneficial to CI users, especially patients with shallow electrode insertion depths.