Estimation of Vowel Recognition With Cochlear Implant Simulations

Effects of spectral smearing on speech understanding and masking release in simulated bilateral cochlear implants

Differences in spectro-temporal degradation may explain some variability in cochlear implant users' speech outcomes. The present study employs vocoder simulations on listeners with typical hearing to evaluate how differences in degree of channel interaction across ears affects spatial speech recognition. Speech recognition thresholds and spatial release from masking were measured in 16 normal-hearing subjects listening to simulated bilateral cochlear implants. 16-channel sine-vocoded speech simulated limited, broad, or mixed channel interaction, in dichotic and diotic target-masker conditions, across ears. Thresholds were highest with broad channel interaction in both ears but improved when interaction decreased in one ear and again in both ears. Masking release was apparent across conditions. Results from this simulation study on listeners with typical hearing show that channel interaction may impact speech recognition more than masking release, and may have implications for the effects of channel interaction on cochlear implant users' speech recognition outcomes.

Spectral and Temporal Envelope Cues for Human and Automatic Speech Recognition in Noise

Acoustic features of speech include various spectral and temporal cues. It is known that temporal envelope plays a critical role for speech recognition by human listeners, while automated speech recognition (ASR) heavily relies on spectral analysis. This study compared sentence-recognition scores of humans and an ASR software, Dragon, when spectral and temporal-envelope cues were manipulated in background noise. Temporal fine structure of meaningful sentences was reduced by noise or tone vocoders. Three types of background noise were introduced: a white noise, a time-reversed multi-talker noise, and a fake-formant noise. Spectral information was manipulated by changing the number of frequency channels. With a 20-dB signal-to-noise ratio (SNR) and four vocoding channels, white noise had a stronger disruptive effect than the fake-formant noise. The same observation with 22 channels was made when SNR was lowered to 0 dB. In contrast, ASR was unable to function with four vocoding channels even with a 20-dB SNR. Its performance was least affected by white noise and most affected by the fake-formant noise. Increasing the number of channels, which improved the spectral resolution, generated non-monotonic behaviors for the ASR with white noise but not with colored noise. The ASR also showed highly improved performance with tone vocoders. It is possible that fake-formant noise affected the software's performance by disrupting spectral cues, whereas white noise affected performance by compromising speech segmentation. Overall, these results suggest that human listeners and ASR utilize different listening strategies in noise.

The effects of noise vocoding on gap detection thresholds

Gap detection threshold (GDT), the shortest silent interval a person can perceive, is a commonly used measure of temporal processing resolution. The purposes of this study were: (1) to examine the effects of noise vocoding, which has been used to simulate what signals sound like through a cochlear implant, on GDTs in normal-hearing subjects, and (2) to further the understanding of neural mechanisms underlying gap detection using the Auditory Late Response (ALR). Thirteen normal listeners participated. In behavioral tests, the GDTs were determined for the original and vocoded stimuli. In ALR recordings, the subjects were presented with auditory stimuli with and without containing gaps and stimuli with and without being vocoded. Results showed that GDTs were significantly elevated for vocoded stimuli with spectral resolutions of 4 and 20 channels compared to those for the original stimuli. A gap effect was observed in the post-gap ALR. Current densities for N1 peaks evoked by stimuli with zero- vs. non-zero ms gaps, pre- vs. post-gap markers, and original vs. vocoded stimuli were obtained using the standardized low-resolution brain electromagnetic tomography (sLORETA) method. Paired comparisons of pre- and post-gap current density values were made. Results showed a statistical difference between the N1s evoked by pre- vs. post-gap markers, with the activation in the middle frontal gyrus and precentral gyrus. The results suggest that: (1) noise vocoding does affect temporal processing resolution assessed with GDTs, (2) gap detection may involve the recruitment of cognitive neural resources, and (3) the ALR has a potential value of objectively estimating temporal processing resolution.

Effects of spectral modulation filtering on vowel identification

The goal of this study was to measure the effects of global spectral manipulations on vowel identification by progressively high-pass filtering vowel stimuli in the spectral modulation domain. Twelve American-English vowels, naturally spoken by a female talker, were subjected to varied degrees of high-pass filtering in the spectral modulation domain, with cutoff frequencies of 0.0, 0.5, 1.0, 1.5, and 2.0 cycles/octave. Identification performance for vowels presented at 70 dB sound pressure level with and without spectral modulation filtering was measured for five normal-hearing listeners. Results indicated that vowel identification performance was progressively degraded as the spectral modulation cutoff frequency increased. Degradation of vowel identification was greater for back vowels than for front or central vowels. Detailed acoustic analyses indicated that spectral modulation filtering resulted in a more crowded vowel space (F1xF2), reduced spectral contrast, and reduced spectral tilt relative to the original unfiltered vowels. Changes in the global spectral features produced by spectral modulation filtering were associated with substantial reduction in vowel identification. The results indicated that the spectral cues critical for vowel identification were represented by spectral modulation frequencies below 2 cycles/octave. These results are considered in terms of the interactions among spectral shape perception, spectral smearing, and speech perception.

Perceptual adaptation to spectrally shifted vowels: training with nonlexical labels

Although normal-hearing (NH) and cochlear implant (CI) listeners are able to adapt to spectrally shifted speech to some degree, auditory training has been shown to provide more complete and/or accelerated adaptation. However, it is unclear whether listeners use auditory and visual feedback to improve discrimination of speech stimuli, or to learn the identity of speech stimuli. The present study investigated the effects of training with lexical and nonlexical labels on NH listeners' perceptual adaptation to spectrally degraded and spectrally shifted vowels. An eight-channel sine wave vocoder was used to simulate CI speech processing. Two degrees of spectral shift (moderate and severe shift) were studied with three training paradigms, including training with lexical labels (i.e., "hayed," "had," "who'd," etc.), training with nonlexical labels (i.e., randomly assigned letters "f," "b," "g," etc.), and repeated testing with lexical labels (i.e., "test-only" paradigm without feedback). All training and testing was conducted over 5 consecutive days, with two to four training exercises per day. Results showed that with the test-only paradigm, lexically labeled vowel recognition significantly improved for moderately shifted vowels; however, there was no significant improvement for severely shifted vowels. Training with nonlexical labels significantly improved the recognition of nonlexically labeled vowels for both shift conditions; however, this improvement failed to generalize to lexically labeled vowel recognition with severely shifted vowels. Training with lexical labels significantly improved lexically labeled vowel recognition with severely shifted vowels. These results suggest that storage and retrieval of speech patterns in the central nervous system is somewhat robust to tonotopic distortion and spectral degradation. Although training with nonlexical labels may improve discrimination of spectrally distorted peripheral patterns, lexically meaningful feedback is needed to identify these peripheral patterns. The results also suggest that training with lexically meaningful feedback may be beneficial to CI users, especially patients with shallow electrode insertion depths.