Adaptation by normal listeners to upward spectral shifts of speech: implications for cochlear implants

Adaptation to distorted frequency-to-place maps: implications of simulations in normal listeners for cochlear implants and electroacoustic stimulation

The ideal cochlear implant electrode array positioning enables stimulation over a range of cochlear positions whose characteristic frequencies cover the frequency range of speech and match the speech processor filter frequencies. However, the electrode positions achieved in practice may not meet this specification. Users of conventional monaural cochlear implants seem able to perceptually adapt to a mismatch of speech processor filters to electrode positions. In electroacoustic stimulation, it is important to consider possible inconsistencies between acoustic and electrical frequency-to-place mapping. Two simulation studies are outlined that address normal listeners' ability to perceive speech presented through distorted frequency maps. The first presented a map that is spectrally warped around a 10-mm medial cochlear area. Listeners were able to adapt to this map after a few hours of training. The second study presented a binaural mapping in which one ear was subject to a 6-mm basalward shift. Here listeners were unable to learn to integrate speech information across the two mismatched ears, rather they seem to learn to ignore the shifted information. Frequency-to-place mapping is likely to be an important factor in the successful use of a combination of electrical and acoustic hearing.

The right information may matter more than frequency-place alignment: simulations of frequency-aligned and upward shifting cochlear implant processors for a shallow electrode array insertion

OBJECTIVE

It has been claimed that speech recognition with a cochlear implant is dependent on the correct frequency alignment of analysis bands in the speech processor with characteristic frequencies (CFs) at electrode locations. However, the use of filters aligned in frequency to a relatively basal electrode array position leads to significant loss of lower frequency speech information. This study uses an acoustic simulation to compare two approaches to the matching of speech processor filters to an electrode array having a relatively shallow depth within the typical range, such that the most apical element is at a CF of 1851 Hz. Two noise-excited vocoder speech processors are compared, one with CF-matched filters, and one with filters matched to CFs at basilar membrane locations 6 mm more apical than electrode locations.

DESIGN

An extended crossover training design examined pre- and post-training performance in the identification of vowels and words in sentences for both processors. Subjects received about 3 hours of training with each processor in turn.

RESULTS

Training improved performance with both processors, but training effects were greater for the shifted processor. For a male talker, the shifted processor led to higher post-training scores than the frequency-aligned processor with both vowels and sentences. For a female talker, post-training vowel scores did not differ significantly between processors, whereas sentence scores were higher with the frequency-aligned processor.

CONCLUSIONS

Even for a shallow electrode insertion, we conclude that a speech processor should represent information from important frequency regions below 1 kHz and that the possible cost of frequency misalignment can be significantly reduced with listening experience.

Dissociations in perceptual learning revealed by adult age differences in adaptation to time-compressed speech

When presented with several time-compressed sentences, young adults' performance improves with practice. Such adaptation has not been studied in older adults. To study age-related changes in perceptual learning, the authors tested young and older adults' ability to adapt to degraded speech. First, the authors showed that older adults, when equated for starting accuracy with young adults, adapted at a rate and magnitude comparable to young adults. However, unlike young adults, older adults failed to transfer this learning to a different speech rate and did not show additional benefit when practice exceeded 20 sentences. Listeners did not adapt to speech degraded by noise, indicating that adaptation to time-compressed speech was not attributable to task familiarity. Finally, both young and older adults adapted to spectrally shifted noise-vocoded speech. The authors conclude that initial perceptual learning is comparable in young and older adults but maintenance and transfer of this learning decline with age.

Effects of envelope-vocoder processing on F0 discrimination and concurrent-vowel identification

OBJECTIVE

The aim of this study was to examine the effects of envelope-vocoder sound processing on listeners' ability to discriminate changes in fundamental frequency (F0) in anechoic and reverberant conditions and on their ability to identify concurrent vowels based on differences in F0.

DESIGN

In the first experiment, F0 difference limens (F0DLs) were measured as a function of number of envelope-vocoder frequency channels (1, 4, 8, 24, and 40 channels, and unprocessed) in four normal-hearing listeners, with degree of simulated reverberation (no, mild, and severe reverberation) as a parameter. In the second experiment, vowel identification was measured as a function of the F0 difference between two simultaneous vowels in six normal-hearing listeners, with the number of vocoder channels (8 and 24 channels, and unprocessed) as a parameter.

RESULTS

Reverberation was detrimental to F0 discrimination in conditions with fewer numbers of vocoder channels. Despite the reasonable F0DLs (<1 semitone) with 24- and 8-channel vocoder processing, listeners were unable to benefit from F0 differences between the competing vowels in the concurrent-vowel paradigm.

CONCLUSIONS

The overall detrimental effects of vocoder processing are probably are due to the poor spectral representation of the lower-order harmonics. The F0 information carried in the temporal envelope is weak, susceptible to reverberation, and may not suffice for source segregation. To the extent that vocoder processing simulates cochlear implant processing, users of current implant processing schemes are unlikely to benefit from F0 differences between competing talkers when listening to speech in complex environments. The results provide further incentive for finding a way to make the information from low-order, resolved harmonics available to cochlear implant users.

Voice gender identification by cochlear implant users: the role of spectral and temporal resolution

The present study explored the relative contributions of spectral and temporal information to voice gender identification by cochlear implant users and normal-hearing subjects. Cochlear implant listeners were tested using their everyday speech processors, while normal-hearing subjects were tested under speech processing conditions that simulated various degrees of spectral resolution, temporal resolution, and spectral mismatch. Voice gender identification was tested for two talker sets. In Talker Set 1, the mean fundamental frequency values of the male and female talkers differed by 100 Hz while in Talker Set 2, the mean values differed by 10 Hz. Cochlear implant listeners achieved higher levels of performance with Talker Set 1, while performance was significantly reduced for Talker Set 2. For normal-hearing listeners, performance was significantly affected by the spectral resolution, for both Talker Sets. With matched speech, temporal cues contributed to voice gender identification only for Talker Set 1 while spectral mismatch significantly reduced performance for both Talker Sets. The performance of cochlear implant listeners was similar to that of normal-hearing subjects listening to 4-8 spectral channels. The results suggest that, because of the reduced spectral resolution, cochlear implant patients may attend strongly to periodicity cues to distinguish voice gender.

Auditory training with spectrally shifted speech: implications for cochlear implant patient auditory rehabilitation

After implantation, postlingually deafened cochlear implant (CI) patients must adapt to both spectrally reduced and spectrally shifted speech, due to the limited number of electrodes and the limited length of the electrode array. This adaptation generally occurs during the first three to six months of implant use and may continue for many years. To see whether moderate speech training can accelerate this learning process, 16 naïve, normal-hearing listeners were trained with spectrally shifted speech via an eight-channel acoustic simulation of CI speech processing. Baseline vowel and consonant recognition was measured for both spectrally shifted and unshifted speech. Short daily training sessions were conducted over five consecutive days, using four different protocols. For the test-only protocol, no improvement was seen over the five-day period. Similarly, sentence training provided little benefit for vowel recognition. However, after five days of targeted phoneme training, subjects' recognition of spectrally shifted vowels significantly improved in most subjects. This improvement did not generalize to the spectrally unshifted vowel and consonant tokens, suggesting that subjects adapted to the specific spectral shift, rather than to the eight-channel processing in general. Interestingly, significant improvement was also observed for the recognition of spectrally shifted consonants. The largest improvement was observed with targeted vowel contrast training, which did not include any explicit consonant training. These results suggest that targeted phoneme training can accelerate adaptation to spectrally shifted speech. Given these results with normal-hearing listeners, auditory rehabilitation tools that provide targeted phoneme training may be effective in improving the speech recognition performance of adult CI users.

Mandarin tone recognition in cochlear-implant subjects

This study examined tone recognition in five cochlear-implant subjects who were native speakers of Mandarin and used a Nucleus-22 device. Psychophysical experiments were conduced to measure rate discrimination in individual electrodes from the most apical to the most basal electrodes. The rate range was from 100 to 200 Hz, which corresponded to the range of variation in fundamental frequency for the tonal tokens used in this study. Speech recognition experiments were also conducted to measure tone recognition as function of the number of electrodes from a 1-electrode map to a 20-electrode map. Large individual variability was observed for both rate discrimination and tone recognition result: Average rate discrimination ranged between 0.2 and 1.2 (Weber's fraction) whereas tone recognition ranged between 30% and 70% correct. A highly significant correlation was found between rate discrimination and tone recognition with the 20-electrode map, but a non-significant correlation was observed with the 1-electrode map due to a floor effect in tone recognition. The present result supports the hypothesis that both spectral and temporal cues contribute to tone recognition. In addition, the present result shows that current cochlear-implant subjects produced significantly lower performance than acoustic simulations in normal-hearing subjects, suggesting that neither temporal nor spectral cues have been adequately and appropriately extracted and encoded in current cochlear implants. New designs are discussed to improve tone recognition in cochlear implant subjects.

Interactions between cochlear implant electrode insertion depth and frequency-place mapping

While new electrode designs allow deeper insertion and wider coverage in the cochlea, there is still considerable variation in the insertion depth of the electrode array among cochlear implant users. The present study measures speech recognition as a function of insertion depth, varying from a deep insertion of 10 electrodes at 28.8 mm to a shallow insertion of a single electrode at 7.2 mm, in four Med-El Combi 40+ users. Short insertion depths were simulated by inactivating apical electrodes. Speech recognition increased with deeper insertion, reaching an asymptotic level at 21.6 or 26.4 mm depending on the frequency-place map used. Başkent and Shannon [J. Acoust. Soc. Am. 116, 3130-3140 (2004)] showed that speech recognition by implant users was best when the acoustic input frequency was matched onto the cochlear location that normally processes that frequency range, minimizing the spectral distortions in the map. However, if an electrode array is not fully inserted into the cochlea, a matched map will result in the loss of considerable low-frequency information. The results show a strong interaction between the optimal frequency-place mapping and electrode insertion depth. Consistent with previous studies, frequency-place matching produced better speech recognition than compressing the full speech range onto the electrode array for full insertion ranges (20 to 25 mm from the round window). For shallower insertions (16.8 and 19.2 mm) a mild amount of frequency-place compression was better than truncating the frequency range to match the basal cochlear location. These results show that patients with shallow electrode insertions might benefit from a map that assigns a narrower frequency range than patients with full insertions.

Lexical Information Drives Perceptual Learning of Distorted Speech: Evidence From the Comprehension of Noise-Vocoded Sentences

Speech comprehension is resistant to acoustic distortion in the input, reflecting listeners' ability to adjust perceptual processes to match the speech input. For noise-vocoded sentences, a manipulation that removes spectral detail from speech, listeners' reporting improved from near 0% to 70% correct over 30 sentences (Experiment 1). Learning was enhanced if listeners heard distorted sentences while they knew the identity of the undistorted target (Experiments 2 and 3). Learning was absent when listeners were trained with nonword sentences (Experiments 4 and 5), although the meaning of the training sentences did not affect learning (Experiment 5). Perceptual learning of noise-vocoded speech depends on higher level information, consistent with top-down, lexically driven learning. Similar processes may facilitate comprehension of speech in an unfamiliar accent or following cochlear implantation.

Effects of Insertion Depth of Cochlear Implant Electrodes upon Speech Perception

OBJECTIVE

To investigate whether the insertion depth of a cochlear implant array affects postoperative speech perception.

DESIGN

The subjects were 48 postlingually deaf adults who received either the Nucleus 22 or the Nucleus 24 cochlear implant with a straight array. A postoperative radiograph of the cochlear electrode was used to estimate insertion depth, as either the angle of the electrode tip (angle) or the intracochlear length of the electrode (length). Other estimates of insertion depth included the numbers of active electrodes and channels used by the speech processor. Electrode depth, together with the duration of deafness, hearing aid usage, pre-operative speech perception score and pre-operative pure-tone averages were independent variables in a forward stepwise multiple regression analysis, where the dependent variables were postoperative CNC words and CNC phonemes.

RESULTS

Duration of deafness and insertion depth (angle, insertion length or active electrodes) were the predictive variables for CNC words or CNC phonemes. Angle was the best 'depth-related' predictor of postoperative speech perception. An even clearer relationship was found between CUNY sentences in noise and angle, in a subset of 26 patients.

CONCLUSION

Depth of electrode insertion affects postoperative speech perception.

Adaptation by a cochlear-implant patient to upward shifts in the frequency representation of speech

The purpose of this project was to assess the degree to which a patient, after 1 wk of experience, could adapt to 3.2-mm and 6.8-mm basal shifts in the representation of speech. Only small deficits in performance were found after practice after the 3.2-mm shift. After practice after the 6.9-mm shift, scores on tests that emphasized amplitude envelope cues returned to baseline levels. Scores on vowel and sentence tests that emphasized frequency-based cues remained poor. Scores for "place," however, showed some recovery. Vowel recognition may be the limiting factor in recognizing basally shifted speech.

Trends in cochlear implants

More than 60,000 people worldwide use cochlear implants as a means to restore functional hearing. Although individual performance variability is still high, an average implant user can talk on the phone in a quiet environment. Cochlear-implant research has also matured as a field, as evidenced by the exponential growth in both the patient population and scientific publication. The present report examines current issues related to audiologic, clinical, engineering, anatomic, and physiologic aspects of cochlear implants, focusing on their psychophysical, speech, music, and cognitive performance. This report also forecasts clinical and research trends related to presurgical evaluation, fitting protocols, signal processing, and postsurgical rehabilitation in cochlear implants. Finally, a future landscape in amplification is presented that requires a unique, yet complementary, contribution from hearing aids, middle ear implants, and cochlear implants to achieve a total solution to the entire spectrum of hearing loss treatment and management.

Perceptual learning and nonword repetition using a cochlear implant simulation

This study examined the effects of perceptual learning on nonword repetition performance of normal-hearing listeners who were exposed to severely degraded auditory conditions that were designed to simulate the auditory input of a cochlear implant. Twenty normal-hearing adult listeners completed a nonword repetition task using an eight-band, frequency-shifted cochlear implant simulation strategy both before and after training on open- and closed-set word recognition tasks. Feedback was provided during training. The nonword responses obtained from each participant were digitally recorded and played back to normal-hearing listeners. These listeners rated the nonword repetition accuracy in comparison to the original unprocessed target stimuli using a seven-point scale. The mean nonword accuracy ratings were significantly higher for the non words repeated after training than for non words repeated prior to training. These results suggest that the word recognition training tasks encouraged auditory perceptual learning that generalized to novel, nonword auditory stimuli. The present findings also suggest that adaptation and learning from the degraded auditory stimuli produced by a cochlear implant simulation can be achieved even in a difficult perceptual-motor task such as nonword repetition which involves both speech perception and production of an auditory stimulus that lacks any lexical or semantic representation.

Effects of presentation mode and repeated familiarization on intelligibility of dysarthric speech

Clinical measures of speech intelligibility are widely used as one means of characterizing the speech of individuals with dysarthria. Many variables associated with both the speaker and the listener contribute to what is actually measured as intelligibility. The present study explored the effects of presentation modality (audiovisual vs. audio-only information) and the effects of speaker-specific familiarization across 4 trials on the intelligibility of speakers with mild and severe dysarthria associated with cerebral palsy. Results revealed that audiovisual information did not enhance intelligibility relative to audio-only information for 4 of the 5 speakers studied. The one speaker whose intelligibility increased when audiovisual information was presented had the most severe dysarthria and concomitant motor impairments. Results for speakerspecific repeated familiarization were relatively homogeneous across speakers, demonstrating significant intelligibility score improvements across 4 trials and, in particular, a significant improvement in intelligibility between the 1st and 4th trials.

Speech recognition under conditions of frequency-place compression and expansion

In normal acoustic hearing the mapping of acoustic frequency information onto the appropriate cochlear place is a natural biological function, but in cochlear implants it is controlled by the speech processor. The cochlear tonotopic range of the implant is determined by the length and insertion depth of the electrode array. Conventional cochlear implant electrode arrays are designed for an insertion of 25 mm inside the round window and the active electrodes occupy 16 mm, which would place the electrodes in a cochlear region corresponding to an acoustic frequency range of 500-6000 Hz. However, some implant speech processors map an acoustic frequency range from 150 to 10000 Hz onto these electrodes. While this mapping preserves the entire range of acoustic frequency information, it also results in a compression of the tonotopic pattern of speech information delivered to the brain. The present study measured the effects of such a compression of frequency-to-place mapping on speech recognition using acoustic simulations. Also measured were the effects of an expansion of the frequency-to-place mapping, which produces an expanded representation of speech in the cochlea. Such an expanded representation might improve speech recognition by improving the relative spatial (tonotopic) resolution, like an "acoustic fovea." Phoneme and sentence recognition was measured as a function of linear (in terms of cochlear distance) frequency-place compression and expansion. These conditions were presented to normal-hearing listeners using a noise-band vocoder, simulating cochlear implant electrodes with different insertion depths and different number of electrode channels. The cochlear tonotopic range was held constant by employing the same noise carrier bands for each condition, while the analysis frequency range was either compressed or expanded relative to the carrier frequency range. For each condition, the result was compared to that of the perfect frequency-place match, where the carrier and the analysis bands were perfectly matched. Speech recognition in the matched conditions was generally better than any condition of frequency-place expansion and compression, even when the matched condition eliminated a considerable amount of acoustic information. This result suggests that speech recognition, at least without training, is dependent on the mapping of acoustic frequency information onto the appropriate cochlear place.

Coding of sounds in the auditory system and its relevance to signal processing and coding in cochlear implants

OBJECTIVE

To review how the properties of sounds are "coded" in the normal auditory system and to discuss the extent to which cochlear implants can and do represent these codes.

DATA SOURCES

Data are taken from published studies of the response of the cochlea and auditory nerve to simple and complex stimuli, in both the normal and the electrically stimulated ear. REVIEW CONTENT: The review describes: 1) the coding in the normal auditory system of overall level (which partly determines perceived loudness), spectral shape (which partly determines perceived timbre and the identity of speech sounds), periodicity (which partly determines pitch), and sound location; 2) the role of the active mechanism in the cochlea, and particularly the fast-acting compression associated with that mechanism; 3) the neural response patterns evoked by cochlear implants; and 4) how the response patterns evoked by implants differ from those observed in the normal auditory system in response to sound. A series of specific issues is then discussed, including: 1) how to compensate for the loss of cochlear compression; 2) the effective number of independent channels in a normal ear and in cochlear implantees; 3) the importance of independence of responses across neurons; 4) the stochastic nature of normal neural responses; 5) the possible role of across-channel coincidence detection; and 6) potential benefits of binaural implantation.

CONCLUSIONS

Current cochlear implants do not adequately reproduce several aspects of the neural coding of sound in the normal auditory system. Improved electrode arrays and coding systems may lead to improved coding and, it is hoped, to better performance.

The effects of short-term training for spectrally mismatched noise-band speech

The present study examined the effects of short-term perceptual training on normal-hearing listeners' ability to adapt to spectrally altered speech patterns. Using noise-band vocoder processing, acoustic information was spectrally distorted by shifting speech information from one frequency region to another. Six subjects were tested with spectrally shifted sentences after five days of practice with upwardly shifted training sentences. Training with upwardly shifted sentences significantly improved recognition of upwardly shifted speech; recognition of downwardly shifted speech was nearly unchanged. Three subjects were later trained with downwardly shifted speech. Results showed that the mean improvement was comparable to that observed with the upwardly shifted training. In this retrain and retest condition, performance was largely unchanged for upwardly shifted sentence recognition, suggesting that these listeners had retained some of the improved speech perception resulting from the previous training. The results suggest that listeners are able to partially adapt to a spectral shift in acoustic speech patterns over the short-term, given sufficient training. However, the improvement was localized to where the spectral shift was trained, as no change in performance was observed for spectrally altered speech outside of the trained regions.

Simulations of tonotopically mapped speech processors for cochlear implant electrodes varying in insertion depth

It has been claimed that speech recognition with a cochlear implant is dependent on the frequency alignment of analysis bands in the speech processor with characteristic frequencies (CFs) at electrode locations. However, the most apical electrode location can often have a CF of 1 kHz or more. The use of filters aligned in frequency to relatively basal electrode arrays leads to the loss of lower frequency speech information. This study simulates a frequency-aligned speech processor and common array insertion depths to assess this significance of this loss. Noise-excited vocoders simulated processors driving eight electrodes 2 mm apart. Analysis filters always had center frequencies matching the CFs of the simulated stimulation sites. The simulated insertion depth of the most apical electrode was varied in 2-mm steps between 25 mm (CF 502 Hz) and 17 mm (CF 1851 Hz) from the cochlear base. Identification of consonants, vowels, and words in sentences all showed a significant decline between each of the three more basal simulated electrode configurations. Thus, if implant processors used analysis filters frequency-aligned to electrode CFs, patients whose most apical electrode is 19 mm (CF 1.3 kHz) or less from the cochlear base would suffer a significant loss of speech information.

The effects of familiarization on intelligibility and lexical segmentation in hypokinetic and ataxic dysarthria

This study is the third in a series that has explored the source of intelligibility decrement in dysarthria by jointly considering signal characteristics and the cognitive-perceptual processes employed by listeners. A paradigm of lexical boundary error analysis was used to examine this interface by manipulating listener constraints with a brief familiarization procedure. If familiarization allows listeners to extract relevant segmental and suprasegmental information from dysarthric speech, they should obtain higher intelligibility scores than nonfamiliarized listeners, and their lexical boundary error patterns should approximate those obtained in misperceptions of normal speech. Listeners transcribed phrases produced by speakers with either hypokinetic or ataxic dysarthria after being familiarized with other phrases produced by these speakers. Data were compared to those of nonfamiliarized listeners [Liss et al., J. Acoust. Soc. Am. 107, 3415-3424 (2000)]. The familiarized groups obtained higher intelligibility scores than nonfamiliarized groups, and the effects were greater when the dysarthria type of the familiarization procedure matched the dysarthria type of the transcription task. Remarkably, no differences in lexical boundary error patterns were discovered between the familiarized and nonfamiliarized groups. Transcribers of the ataxic speech appeared to have difficulty distinguishing strong and weak syllables in spite of the familiarization. Results suggest that intelligibility decrements arise from the perceptual challenges posed by the degraded segmental and suprasegmental aspects of the signal, but that this type of familiarization process may differentially facilitate mapping segmental information onto existing phonological categories.

Factors that allow a high level of speech understanding by patients fit with cochlear implants

Three factors account for the high level of speech understanding in quiet enjoyed by many patients fit with cochlear implants. First, some information about speech exists in the time/amplitude envelope of speech. This information is sufficient to narrow the number of word candidates for a given signal. Second, if information from the envelope of speech is available to listeners, then only minimal information from the frequency domain is necessary for high levels of speech recognition in quiet. Third, perceiving strategies for speech are inherently flexible in terms of the mapping between signal frequencies (i.e., the locations of the formants) and phonetic identity.

The relative importance of amplitude, temporal, and spectral cues for cochlear implant processor design

Speech understanding with cochlear implants has improved steadily over the last 25 years, and the success of implants has provided a powerful tool for understanding speech recognition in general. Comparing speech recognition in normal-hearing listeners and in cochlear-implant listeners has revealed many important lessons about the types of information necessary for good speech recognition--and some of the lessons are surprising. This paper presents a summary of speech perception research over the last 25 years with cochlear-implant and normal-hearing listeners. As long as the speech is audible, even the relatively severe amplitude distortion has only a mild effect on intelligibility. Temporal cues appear to be useful for speech intelligibility only up to about 20 Hz. Whereas temporal information above 20 Hz may contribute to improved quality, it contributes little to speech understanding. In contrast, the quantity and quality of spectral information appear to be critical for speech understanding. Only four spectral "channels" of information can produce good speech understanding, but more channels are required for difficult listening situations. Speech understanding is sensitive to the placement of spectral information along the cochlea. In prosthetic devices, in which the spectral information can be delivered to any cochlear location, it is critical to present spectral information to the normal acoustic tonotopic location for that information. If there is a shift or distortion of 2 to 3 mm between frequency and cochlear place, speech recognition is decreased dramatically.

Spectral and temporal cues to pitch in noise-excited vocoder simulations of continuous-interleaved-sampling cochlear implants

Four-band and single-band noise-excited vocoders were used in acoustic simulations to investigate spectral and temporal cues to melodic pitch in the output of a cochlear implant speech processor. Noise carriers were modulated by amplitude envelopes extracted by half-wave rectification and low-pass filtering at 32 or 400 Hz. The four-band, but not the single-band processors, may preserve spectral correlates of fundamental frequency (F0). Envelope smoothing at 400 Hz preserves temporal correlates of F0, which are eliminated with 32-Hz smoothing. Inputs to the processors were sawtooth frequency glides, in which spectral variation is completely determined by F0, or synthetic diphthongal vowel glides, whose spectral shape is dominated by varying formant resonances. Normal listeners labeled the direction of pitch movement of the processed stimuli. For processed sawtooth waves, purely temporal cues led to decreasing performance with increasing F0. With purely spectral cues, performance was above chance despite the limited spectral resolution of the processors. For processed diphthongs, performance with purely spectral cues was at chance, showing that spectral envelope changes due to formant movement obscured spectral cues to F0. Performance with temporal cues was poorer for diphthongs than for sawtooths, with very limited discrimination at higher F0. These data suggest that, for speech signals through a typical cochlear implant processor, spectral cues to pitch are likely to have limited utility, while temporal envelope cues may be useful only at low F0.

Perceptual learning following changes in the frequency-to-electrode assignment with the Nucleus-22 cochlear implant

The goal of the present study was to investigate the time course of adaptation by experienced cochlear implant users to a shifted frequency-to-electrode assignment in their speech processors. Speech recognition performance of three Nucleus-22 cochlear implant users was measured over a 3-month period, during which the implant listeners continuously wore "experimental" speech processors that were purposely shifted by 2-4 mm in terms of the frequency-to-electrode assignment relative to their normal processor. Baseline speech performance was measured with each subject's clinically assigned speech processor just prior to implementation of the experimental processor. Baseline speech performance was measured again after the 3-month test period, immediately following the reinstallation of the clinically assigned processor settings. Speech performance with the experimental processor was measured four times during the first week, and weekly thereafter over the 3-month period. Results showed that the experimental processor produced significantly lower performance on all measures of speech recognition immediately following implementation. Over the 3-month test period, consonant and HINT sentence recognition with the experimental processors gradually approached a performance level comparable to but still significantly below the baseline and postexperiment measures made with the clinically assigned processor. However, vowel and TIMIT sentence recognition with the experimental processors remained far below the level of the baseline measures even at the end of the 3-month experimental period. There was no significant change in performance with the clinically assigned processor before or after fitting with the experimental processor. The results suggest that a long-time exposure to a new pattern of stimulation may not be able to compensate for the deficit in performance caused by a 2-4-mm shift in the tonotopic location of stimulation, at least within a 3-month period.

Phoneme recognition and confusions with multichannel cochlear implants: vowels

The aim of this study was to investigate how postlingually severely or profoundly hearing-impaired adults relearn to recognize vowels after receiving multichannel cochlear implants. Vowel recognition of 19 Finnish-speaking subjects was studied for a minimum of 6 months and a maximum of 24 months using an open-set nonsense-syllable test in a prospective repeated-measure design. The responses were coded for phoneme errors, and 95% confidence intervals for recognition and confusions were calculated. The average vowel recognition was 68% (95% confidence interval = 66-70%) 6 months after switch-on and 80% (95% confidence interval = 78-82%) 24 months after switch-on. The vowels [ae], [u], [i], [o], and [a] were the easiest to recognize, and the vowels [y], [e], and [ø] were the most difficult. In conclusion, adaptation to electrical hearing using a multichannel cochlear implant was achieved well; but for at least 2 years, given two vowels with either F1 or F2 at roughly the some frequencies, confusions were drawn more towards the closest vowel with the next highest F1 or F2.

Phoneme recognition and confusions with multichannel cochlear implants: consonants

The aim of this study was to investigate how postlingually severely or profoundly hearing-impaired adults relearn to recognize consonants after receiving multichannel cochlear implants. Consonant recognition of 19 Finnish-speaking subjects was studied for a minimum of 6 months and a maximum of 24 months using an open-set nonsense-syllable test in a prospective repeated-measure design. Responses were coded for phoneme errors, and proportions of correct responses and 95% confidence intervals were calculated for recognition and confusions. Two years after the switch-on, the mean recognition of consonants was 71% (95% confidence interval = 68-73%). The manner of articulation was easier to classify than the place of articulation, and the consonants [s], [r], [k], [t], [p], [n], and [j] were easier to recognize than [h], [m], [l], and [v]. Adaptation to electrical hearing with a multichannel cochlear implant was successful, but consonants with alveolar, palatal, or velar transitions (high F2) were better recognized than consonants with labial transitions (low F2). The locus of the F2 transitions of the consonants with better recognition was at the frequencies 1.5-2 kHz, whereas the locus of the F2 transitions of the consonants with poorer recognition was at 1.2-1.4 kHz. A tendency to confuse consonants with the closest consonant with higher F2 transition was also noted.

Frequency mapping in cochlear implants

OBJECTIVE

To understand the short-term ("acute") effects of parametric variations to the frequency-to-electrode mapping on phoneme identification by Nucleus-22 cochlear implant listeners.

METHODS

Phoneme recognition was measured in five Nucleus-22 cochlear implant listeners using custom four-channel continuous interleaved sampler (CIS) processors. For the four-channel processors, speech signals were band-pass filtered into four broad frequency bands. The temporal envelope in each band was extracted by half-wave rectification and low-pass filtering at 160 Hz. The extracted envelope was then transformed to electric currents by a power function with an exponent of 0.2. The resulting electric currents were delivered to four electrode pairs (18,22), (13,17), (8,12), (3,7). The effect of frequency-to-electrode mapping was investigated by systematically varying the parameters of band-pass filters while fixing the electrode locations. Experiment 1 measured phoneme recognition as a function of the slope of band-pass filters. The slope of band-pass filters varied from 48 dB/octave to 6 dB/octave; the corner frequencies of band-pass filters were not varied. Experiment 2 measured phoneme recognition as a function of the distribution of band-pass filters across a fixed overall frequency range. The frequency divisions of a fixed overall frequency range were systematically varied from a logarithmic to a linear distribution. Experiment 3 measured phoneme recognition as a function of the bandwidth of the band-pass filters. The bandwidth of each filter varied from 0.2 to 2 octaves; the center frequencies for each band were not varied. No practice or feedback was provided for subjects in all experiments.

RESULTS

The slope of the band-pass filters had little effect on both vowel and consonant recognition. A slight performance drop was observed for only the shallowest slope condition (6 dB/octave). In contrast, the distribution of the band-pass filters had a strong effect on vowel recognition but a weak effect on consonant recognition. Best performance was achieved when a logarithmic or near-logarithmic frequency distribution was used to divide the overall frequency range. The bandwidth of the band-pass filters had a moderate effect on both vowel and consonant recognition. Vowel scores dropped significantly when the bandwidth of filters was too broad, whereas consonant scores dropped significantly when a narrower bandwidth was used.

CONCLUSION

Under "acute" testing conditions, phoneme recognition with a four-channel CIS strategy seems to be only mildly affected by the slope of the band-pass filters, but can be significantly affected by the distribution of filters as well as the bandwidth of the filters. Optimal or near-optimal performance can be achieved with a logarithmic frequency distribution. Vowels are more susceptible to broad bandwidths, whereas consonants are more susceptible to narrow bandwidths.

The effect of channel interactions on speech recognition in cochlear implant subjects: predictions from an acoustic model

Acoustic models that produce speech signals with information content similar to that provided to cochlear implant users provide a mechanism by which to investigate the effect of various implant-specific processing or hardware parameters independent of other complicating factors. This study compares speech recognition of normal-hearing subjects listening through normal and impaired acoustic models of cochlear implant speech processors. The channel interactions that were simulated to impair the model were based on psychophysical data measured from cochlear implant subjects and include pitch reversals, indiscriminable electrodes, and forward masking effects. In general, spectral interactions degraded speech recognition more than temporal interactions. These effects were frequency dependent with spectral interactions that affect lower-frequency information causing the greatest decrease in speech recognition, and interactions that affect higher-frequency information having the least impact. The results of this study indicate that channel interactions, quantified psychophysically, affect speech recognition to different degrees. Investigation of the effects that channel interactions have on speech recognition may guide future research whose goal is compensating for psychophysically measured channel interactions in cochlear implant subjects.

Sending sound to the brain

The cochlear implant, a microelectrode array that directly stimulates the auditory nerve, has greatly benefited many individuals with profound deafness. Deaf patients without an intact auditory nerve may be helped by the next generation of auditory prostheses: surface or penetrating auditory brainstem implants that bypass the auditory nerve and directly stimulate auditory processing centers in the brainstem.

Frequency-to-electrode allocation and speech perception with cochlear implants

The hypothesis was investigated that selectively increasing the discrimination of low-frequency information (below 2600 Hz) by altering the frequency-to-electrode allocation would improve speech perception by cochlear implantees. Two experimental conditions were compared, both utilizing ten electrode positions selected based on maximal discrimination. A fixed frequency range (200-10513 Hz) was allocated either relatively evenly across the ten electrodes, or so that nine of the ten positions were allocated to the frequencies up to 2600 Hz. Two additional conditions utilizing all available electrode positions (15-18 electrodes) were assessed: one with each subject's usual frequency-to-electrode allocation; and the other using the same analysis filters as the other experimental conditions. Seven users of the Nucleus CI22 implant wore processors mapped with each experimental condition for 2-week periods away from the laboratory, followed by assessment of perception of words in quiet and sentences in noise. Performance with both ten-electrode maps was significantly poorer than with both full-electrode maps on at least one measure. Performance with the map allocating nine out of ten electrodes to low frequencies was equivalent to that with the full-electrode maps for vowel perception and sentences in noise, but was worse for consonant perception. Performance with the evenly allocated ten-electrode map was equivalent to that with the full-electrode maps for consonant perception, but worse for vowel perception and sentences in noise. Comparison of the two full-electrode maps showed that subjects could fully adapt to frequency shifts up to ratio changes of 1.3, given 2 weeks' experience. Future research is needed to investigate whether speech perception may be improved by the manipulation of frequency-to-electrode allocation in maps which have a full complement of electrodes in Nucleus implants.

Use of vocalic information in the identification of /s/ and /sh/ by children with cochlear implants

OBJECTIVE

When a syllable such as "sea" or "she" is spoken, listeners with normal hearing extract evidence of the fricative consonant from both the fricative noise and the following vocalic segment. If the fricative noise is made ambiguous, listeners may still perceive "s" or "sh" categorically, depending on information in the vocalic segment. Do children whose auditory experience comes from electrical stimulation also display this effect, in which a subsequent segment of speech disambiguates an earlier segment?

DESIGN

Unambiguous vowels were appended to ambiguous fricative noises to form tokens of the words "she," "sea," "shoe," and "Sue." A four-choice identification test was undertaken by children with normal hearing (N = 29), prelingually deaf children with the Nucleus Spectra-22 implant system using the SPEAK coding strategy (N = 13), postlingually deafened adults with the same implant system (N = 26), and adults with normal hearing (N = 10). The last group undertook the test before and after the stimuli were processed to simulate the transformations introduced by the SPEAK coding strategy.

RESULTS

All four groups made use of vocalic information. Simulated processing reduced the use made by normal-hearing adults. Implanted subjects made less use than the other groups, with no significant difference between implanted children and implanted adults. The highest levels of use by implanted subjects were within one standard deviation of the mean level displayed when normal-hearing adults listened to processed stimuli. Analyses showed that the SPEAK strategy distorted formant contours in the vocalic segments of the stimuli in ways that are compatible with the errors of identification made by implanted subjects.

CONCLUSIONS

Some children with implants can extract information from a following vowel to disambiguate a preceding fricative noise. The upper limit on this ability may be set by distortions introduced by the implant processor, rather than by the auditory experience of the child.

Effects of the number of channels and speech-to-noise ratio on rate of connected discourse tracking through a simulated cochlear implant speech processor

OBJECTIVE

To investigate the effects of number of channels and speech-to-noise ratio on connected discourse tracking (CDT) through simulations of cochlear implant speech processing. Previous studies have used citation-form vowel and consonant materials or simple sentences. CDT rates were expected to be less likely to be limited by ceiling effects and more representative of everyday speech communication.

DESIGN

Four normal-hearing subjects were presented with speech processed through a real-time sine-excited vocoder having three, four, eight, or 12 channels. Amplitude envelopes extracted from each band modulated sinusoidal carrier signals placed at each band center frequency. Speech-spectrum shaped noise was added to speech before vocoder processing at three signal to noise ratios based on real-time measurements of speech level (+7, +12, +17 dB).

RESULTS

CDT rates increased significantly with number of channels up to eight in both quiet and noise, and decreased significantly with each increase in noise level from quiet.

CONCLUSIONS

The effects on CDT rates of the number of channels and speech-to-noise ratio are highly correlated with intelligibility measures for Hearing in Noise Test (HINT) sentences, consonants and vowels. However, HINT sentence scores even in noise show ceiling effects that obscure the advantages of processors with eight or more channels. Moderate levels of noise that have only slight effects on other measures significantly affected CDT rate. CDT rates with three or four bands of spectral information were much lower than asymptotic rates, especially in the presence of noise.

Effects of low-pass filtering on the intelligibility of speech in quiet for people with and without dead regions at high frequencies

A dead region is a region of the cochlea where there are no functioning inner hair cells (IHCs) and/or neurons; it can be characterized in terms of the characteristic frequencies of the IHCs bordering that region. We examined the effect of high-frequency amplification on speech perception for subjects with high-frequency hearing loss with and without dead regions. The limits of any dead regions were defined by measuring psychophysical tuning curves and were confirmed using the TEN test described in Moore et al. [Br. J. Audiol. 34, 205-224 (2000)]. The speech stimuli were vowel-consonant-vowel (VCV) nonsense syllables, using one of three vowels (/i/, /a/, and /u/) and 21 different consonants. In a baseline condition, subjects were tested using broadband stimuli with a nominal input level of 65 dB SPL. Prior to presentation via Sennheiser HD580 earphones, the stimuli were subjected to the frequency-gain characteristic prescribed by the "Cambridge" formula, which is intended to give speech at 65 dB SPL the same overall loudness as for a normal listener, and to make the average loudness of the speech the same for each critical band over the frequency range important for speech intelligibility (in a listener without a dead region). The stimuli for all other conditions were initially subjected to this same frequency-gain characteristic. Then, the speech was low-pass filtered with various cutoff frequencies. For subjects without dead regions, performance generally improved progressively with increasing cutoff frequency. This indicates that they benefited from high-frequency information. For subjects with dead regions, two patterns of performance were observed. For most subjects, performance improved with increasing cutoff frequency until the cutoff frequency was somewhat above the estimated edge frequency of the dead region, but hardly changed with further increases. For a few subjects, performance initially improved with increasing cutoff frequency and then worsened with further increases, although the worsening was significant only for one subject. The results have important implications for the fitting of hearing aids.

Perceptual "vowel spaces" of cochlear implant users: implications for the study of auditory adaptation to spectral shift

Cochlear implant (CI) users differ in their ability to perceive and recognize speech sounds. Two possible reasons for such individual differences may lie in their ability to discriminate formant frequencies or to adapt to the spectrally shifted information presented by cochlear implants, a basalward shift related to the implant's depth of insertion in the cochlea. In the present study, we examined these two alternatives using a method-of-adjustment (MOA) procedure with 330 synthetic vowel stimuli varying in F1 and F2 that were arranged in a two-dimensional grid. Subjects were asked to label the synthetic stimuli that matched ten monophthongal vowels in visually presented words. Subjects then provided goodness ratings for the stimuli they had chosen. The subjects' responses to all ten vowels were used to construct individual perceptual "vowel spaces." If CI users fail to adapt completely to the basalward spectral shift, then the formant frequencies of their vowel categories should be shifted lower in both F1 and F2. However, with one exception, no systematic shifts were observed in the vowel spaces of CI users. Instead, the vowel spaces differed from one another in the relative size of their vowel categories. The results suggest that differences in formant frequency discrimination may account for the individual differences in vowel perception observed in cochlear implant users.

Effects of the salience of pitch and periodicity information on the intelligibility of four-channel vocoded speech: implications for cochlear implants

Recent simulations of continuous interleaved sampling (CIS) cochlear implant speech processors have used acoustic stimulation that provides only weak cues to pitch, periodicity, and aperiodicity, although these are regarded as important perceptual factors of speech. Four-channel vocoders simulating CIS processors have been constructed, in which the salience of speech-derived periodicity and pitch information was manipulated. The highest salience of pitch and periodicity was provided by an explicit encoding, using a pulse carrier following fundamental frequency for voiced speech, and a noise carrier during voiceless speech. Other processors included noise-excited vocoders with envelope cutoff frequencies of 32 and 400 Hz. The use of a pulse carrier following fundamental frequency gave substantially higher performance in identification of frequency glides than did vocoders using envelope-modulated noise carriers. The perception of consonant voicing information was improved by processors that preserved periodicity, and connected discourse tracking rates were slightly faster with noise carriers modulated by envelopes with a cutoff frequency of 400 Hz compared to 32 Hz. However, consonant and vowel identification, sentence intelligibility, and connected discourse tracking rates were generally similar through all of the processors. For these speech tasks, pitch and periodicity beyond the weak information available from 400 Hz envelope-modulated noise did not contribute substantially to performance.