Comparing live to recorded speech in training the perception of spectrally shifted noise-vocoded speech

The Impact of Spectral and Temporal Degradation on Vocoded Speech Recognition in Early-Blind Individuals

This study compared the impact of spectral and temporal degradation on vocoded speech recognition between early-blind and sighted subjects. The participants included 25 early-blind subjects (30.32 ± 4.88 years; male:female, 14:11) and 25 age- and sex-matched sighted subjects. Tests included monosyllable recognition in noise at various signal-to-noise ratios (-18 to -4 dB), matrix sentence-in-noise recognition, and vocoded speech recognition with different numbers of channels (4, 8, 16, and 32) and temporal envelope cutoff frequencies (50 vs 500 Hz). Cortical-evoked potentials (N2 and P3b) were measured in response to spectrally and temporally degraded stimuli. The early-blind subjects displayed superior monosyllable and sentence recognition than sighted subjects (all p < 0.01). In the vocoded speech recognition test, a three-way repeated-measure analysis of variance (two groups × four channels × two cutoff frequencies) revealed significant main effects of group, channel, and cutoff frequency (all p < 0.001). Early-blind subjects showed increased sensitivity to spectral degradation for speech recognition, evident in the significant interaction between group and channel (p = 0.007). N2 responses in early-blind subjects exhibited shorter latency and greater amplitude in the 8-channel (p = 0.022 and 0.034, respectively) and shorter latency in the 16-channel (p = 0.049) compared with sighted subjects. In conclusion, early-blind subjects demonstrated speech recognition advantages over sighted subjects, even in the presence of spectral and temporal degradation. Spectral degradation had a greater impact on speech recognition in early-blind subjects, while the effect of temporal degradation was similar in both groups.

Effect of spectral degradation on speech intelligibility and cortical representation

Noise-vocoded speech has long been used to investigate how acoustic cues affect speech understanding. Studies indicate that reducing the number of spectral channel bands diminishes speech intelligibility. Despite previous studies examining the channel band effect using earlier event-related potential (ERP) components, such as P1, N1, and P2, a clear consensus or understanding remains elusive. Given our hypothesis that spectral degradation affects higher-order processing of speech understanding beyond mere perception, we aimed to objectively measure differences in higher-order abilities to discriminate or interpret meaning. Using an oddball paradigm with speech stimuli, we examined how neural signals correlate with the evaluation of speech stimuli based on the number of channel bands measuring N2 and P3b components. In 20 young participants with normal hearing, we measured speech intelligibility and N2 and P3b responses using a one-syllable task paradigm with animal and non-animal stimuli across four vocoder conditions with 4, 8, 16, or 32 channel bands. Behavioral data from word repetition clearly affected the number of channel bands, and all pairs were significantly different (p < 0.001). We also observed significant effects of the number of channels on the peak amplitude [F(2.006, 38.117) = 9.077, p < 0.001] and peak latency [F(3, 57) = 26.642, p < 0.001] of the N2 component. Similarly, the P3b component showed significant main effects of the number of channel bands on the peak amplitude [F(2.231, 42.391) = 13.045, p < 0.001] and peak latency [F(3, 57) = 2.968, p = 0.039]. In summary, our findings provide compelling evidence that spectral channel bands profoundly influence cortical speech processing, as reflected in the N2 and P3b components, a higher-order cognitive process. We conclude that spectrally degraded one-syllable speech primarily affects cortical responses during semantic integration.

Preliminary evaluation of computer-assisted home training for French cochlear implant recipients

For French cochlear implant (CI) recipients, in-person clinical auditory rehabilitation is typically provided during the first few years post-implantation. However, this is often inconvenient, it requires substantial time resources and can be problematic when appointments are unavailable. In response, we developed a computer-based home training software ("French AngelSound™") for French CI recipients. We recently conducted a pilot study to evaluate the newly developed French AngelSound™ in 15 CI recipients (5 unilateral, 5 bilateral, 5 bimodal). Outcome measures included phoneme recognition in quiet and sentence recognition in noise. Unilateral CI users were tested with the CI alone. Bilateral CI users were tested with each CI ear alone to determine the poorer ear to be trained, as well as with both ears (binaural performance). Bimodal CI users were tested with the CI ear alone, and with the contralateral hearing aid (binaural performance). Participants trained at home over a one-month period (10 hours total). Phonemic contrast training was used; the level of difficulty ranged from phoneme discrimination in quiet to phoneme identification in multi-talker babble. Unilateral and bimodal CI users trained with the CI alone; bilateral CI users trained with the poorer ear alone. Outcomes were measured before training (pre-training), immediately after training was completed (post-training), and one month after training was stopped (follow-up). For all participants, post-training CI-only vowel and consonant recognition scores significantly improved after phoneme training with the CI ear alone. For bilateral and bimodal CI users, binaural vowel and consonant recognition scores also significantly improved after training with a single CI ear. Follow-up measures showed that training benefits were largely retained. These preliminary data suggest that the phonemic contrast training in French AngelSound™ may significantly benefit French CI recipients and may complement clinical auditory rehabilitation, especially when in-person visits are not possible.

Individual Variability in Recalibrating to Spectrally Shifted Speech: Implications for Cochlear Implants

OBJECTIVES

Cochlear implant (CI) recipients are at a severe disadvantage compared with normal-hearing listeners in distinguishing consonants that differ by place of articulation because the key relevant spectral differences are degraded by the implant. One component of that degradation is the upward shifting of spectral energy that occurs with a shallow insertion depth of a CI. The present study aimed to systematically measure the effects of spectral shifting on word recognition and phoneme categorization by specifically controlling the amount of shifting and using stimuli whose identification specifically depends on perceiving frequency cues. We hypothesized that listeners would be biased toward perceiving phonemes that contain higher-frequency components because of the upward frequency shift and that intelligibility would decrease as spectral shifting increased.

DESIGN

Normal-hearing listeners (n = 15) heard sine wave-vocoded speech with simulated upward frequency shifts of 0, 2, 4, and 6 mm of cochlear space to simulate shallow CI insertion depth. Stimuli included monosyllabic words and /b/-/d/ and /∫/-/s/ continua that varied systematically by formant frequency transitions or frication noise spectral peaks, respectively. Recalibration to spectral shifting was operationally defined as shifting perceptual acoustic-phonetic mapping commensurate with the spectral shift. In other words, adjusting frequency expectations for both phonemes upward so that there is still a perceptual distinction, rather than hearing all upward-shifted phonemes as the higher-frequency member of the pair.

RESULTS

For moderate amounts of spectral shifting, group data suggested a general "halfway" recalibration to spectral shifting, but individual data suggested a notably different conclusion: half of the listeners were able to recalibrate fully, while the other halves of the listeners were utterly unable to categorize shifted speech with any reliability. There were no participants who demonstrated a pattern intermediate to these two extremes. Intelligibility of words decreased with greater amounts of spectral shifting, also showing loose clusters of better- and poorer-performing listeners. Phonetic analysis of word errors revealed certain cues were more susceptible to being compromised due to a frequency shift (place and manner of articulation), while voicing was robust to spectral shifting.

CONCLUSIONS

Shifting the frequency spectrum of speech has systematic effects that are in line with known properties of speech acoustics, but the ensuing difficulties cannot be predicted based on tonotopic mismatch alone. Difficulties are subject to substantial individual differences in the capacity to adjust acoustic-phonetic mapping. These results help to explain why speech recognition in CI listeners cannot be fully predicted by peripheral factors like electrode placement and spectral resolution; even among listeners with functionally equivalent auditory input, there is an additional factor of simply being able or unable to flexibly adjust acoustic-phonetic mapping. This individual variability could motivate precise treatment approaches guided by an individual's relative reliance on wideband frequency representation (even if it is mismatched) or limited frequency coverage whose tonotopy is preserved.

The effect of dopamine on the comprehension of spectrally-shifted noise-vocoded speech: a pilot study

Objectives: Cochlear implantation has proven beneficial in restoring hearing. However, success is variable, and there is a need for a simple post-implantation therapy that could significantly increase implantation success. Dopamine has a general role in learning and in assigning value to environmental stimuli. We tested the effect of dopamine in the comprehension of spectrally-shifted noise-vocoded (SSNV) speech, which simulates, in hearing individuals, the signal delivered by a cochlear implant (CI).Design and study sample: Thirty-five participants (age = 38.0 ± 10.1 SD) recruited from the general population were divided into three groups. We tested SSNV speech comprehension in two experimental sessions. In one session, a metabolic precursor of dopamine (L-DOPA) was administered to participants in two of the groups; a placebo was administered in the other session.Results: A single dose of L-DOPA interacted with training to improve perception of SSNV speech, but did not significantly accelerate learning.Conclusions: These findings are a first step in exploring the use of dopamine to enhance speech understanding in CI patients. Replications of these results using SSNV in individuals with normal hearing, and also in CI users, are needed to determine whether these effects can translate into benefits in everyday language comprehension.

Computer-Based Connected-Text Training of Speech-in-Noise Perception for Cochlear Implant Users

An interactive method for training speech perception in noise was assessed with adult cochlear implant users. The method employed recordings of connected narratives divided into phrases of 4 to 10 words, presented in babble. After each phrase, the listener identified key words from the phrase from among similar sounding foil words. Nine postlingually deafened adult cochlear implant users carried out 12 hr of training over a 4-week period. Training was carried out at home on tablet computers. The primary outcome measure was sentence recognition in babble. Vowel and consonant identification in speech-shaped noise were also assessed, along with digit span in noise, intended as a measure of some important underlying cognitive abilities. Talkers for speech tests were different from those used in training. To control for procedural learning, the test battery was administered repeatedly prior to training. Performance was assessed immediately after training and again after a further 4 weeks during which no training occurred. Sentence recognition in babble improved significantly after training, with an improvement in speech reception threshold of approximately 2 dB, which was maintained at the 4-week follow-up. There was little evidence of improvement in the other measures. It appears that the method has potential as a clinical intervention. However, the underlying sources of improvement and the extent to which benefits generalize to real-world situations remain to be determined.

Teaching language components to deaf/hard-of-hearing and cochlear implant users: a literature review

ABSTRACT Cochlear implants are one of the possible ways for Deaf or Hard-of-Hearing (DHH) individuals to detect sounds and as a mean of insertion in the social, academic and work environments. Nevertheless, in many cases, the cochlear implant alone is not sufficient, demanding hearing and expressive language skills rehabilitation to optimize the device used. This study aimed at reviewing scientific articles that described empirical research with interventions and/or teaching methods of various language repertoires to deaf and hard-of-hearing cochlear implant users. The review was carried out in five scientific databases considering all periods. On the first phase, 156 articles were identified and from these, 24 publications were selected. After being read, these articles were categorized and analyzed as to the participants, teaching targets and procedures adopted. On the last stage of the research, 10 experimental studies were selected and analyzed regarding procedures and results, indicating important factors in teaching this population. The results point to the necessity of an increase of scientific production in the construction and evaluation of effective verbal repertoires teaching programs for cochlear implant users.

Suprasegmental Characteristics of Spontaneous Speech Produced in Good and Challenging Communicative Conditions by Talkers Aged 9-14 Years

PURPOSE

This study investigated the acoustic characteristics of spontaneous speech by talkers aged 9-14 years and their ability to adapt these characteristics to maintain effective communication when intelligibility was artificially degraded for their interlocutor.

METHOD

Recordings were made for 96 children (50 female participants, 46 male participants) engaged in a problem-solving task with a same-sex friend; recordings for 20 adults were used as reference. The task was carried out in good listening conditions (normal transmission) and in degraded transmission conditions. Articulation rate, median fundamental frequency (f0), f0 range, and relative energy in the 1- to 3-kHz range were analyzed.

RESULTS

With increasing age, children significantly reduced their median f0 and f0 range, became faster talkers, and reduced their mid-frequency energy in spontaneous speech. Children produced similar clear speech adaptations (in degraded transmission conditions) as adults, but only children aged 11-14 years increased their f0 range, an unhelpful strategy not transmitted via the vocoder. Changes made by children were consistent with a general increase in vocal effort.

CONCLUSION

Further developments in speech production take place during later childhood. Children use clear speech strategies to benefit an interlocutor facing intelligibility problems but may not be able to attune these strategies to the same degree as adults.

A cross-dialectal acoustic comparison of vowels in Northern and Southern British English

This study compares the duration and first two formants (F1 and F2) of 11 nominal monophthongs and five nominal diphthongs in Standard Southern British English (SSBE) and a Northern English dialect. F1 and F2 trajectories were fitted with parametric curves using the discrete cosine transform (DCT) and the zeroth DCT coefficient represented formant trajectory means and the first DCT coefficient represented the magnitude and direction of formant trajectory change to characterize vowel inherent spectral change (VISC). Cross-dialectal comparisons involving these measures revealed significant differences for the phonologically back monophthongs /ɒ, ɔː, ʊ, uː/ and also /зː/ and the diphthongs /eɪ, əʊ, aɪ, ɔɪ/. Most cross-dialectal differences are in zeroth DCT coefficients, suggesting formant trajectory means tend to characterize such differences, while first DCT coefficient differences were more numerous for diphthongs. With respect to VISC, the most striking differences are that /uː/ is considerably more diphthongized in the Northern dialect and that the F2 trajectory of /əʊ/ proceeds in opposite directions in the two dialects. Cross-dialectal differences were found to be largely unaffected by the consonantal context in which the vowels were produced. The implications of the results are discussed in relation to VISC, consonantal context effects and speech perception.

Adaptation to nonlinear frequency compression in normal-hearing adults: a comparison of training approaches

OBJECTIVE

To identify which training approach, if any, is most effective for improving perception of frequency-compressed speech.

DESIGN

A between-subject design using repeated measures.

STUDY SAMPLE

Forty young adults with normal hearing were randomly allocated to one of four groups: a training group (sentence or consonant) or a control group (passive exposure or test-only). Test and training material differed in terms of material and speaker.

RESULTS

On average, sentence training and passive exposure led to significantly improved sentence recognition (11.0% and 11.7%, respectively) compared with the consonant training group (2.5%) and test-only group (0.4%), whilst, consonant training led to significantly improved consonant recognition (8.8%) compared with the sentence training group (1.9%), passive exposure group (2.8%), and test-only group (0.8%).

CONCLUSIONS

Sentence training led to improved sentence recognition, whilst consonant training led to improved consonant recognition. This suggests learning transferred between speakers and material but not stimuli. Passive exposure to sentence material led to an improvement in sentence recognition that was equivalent to gains from active training. This suggests that it may be possible to adapt passively to frequency-compressed speech.

Overlapping frequency coverage and simulated spatial cue effects on bimodal (electrical and acoustical) sentence recognition in noise

Sentence recognition in 20-talker babble was measured in eight Nucleus cochlear implant (CI) users with contralateral residual acoustic hearing. Speech reception thresholds (SRTs) were measured both in standard configurations, with some frequency regions presented both acoustically and electrically, and in configurations with no spectral overlap. In both cases a continuous interleaved sampling strategy was used. Mean SRTs were around 3 dB better with bimodal presentation than with CI alone in overlap configurations. A spherical head model was used to simulate azimuthal separation of speech and noise and provided no evidence of a contribution of spatial cues to bimodal benefit. There was no effect on bimodal performance of whether spectral overlap was present or was eliminated by switching off electrodes assigned to frequencies below the upper limit of acoustic hearing. In a subsequent experiment the CI was acutely re-mapped so that all available electrodes were used to cover frequencies not presented acoustically. This gave increased spectral resolution via the CI as assessed by formant frequency discrimination, but no improvement in bimodal performance compared to the configuration with overlap.

Simulating the effect of interaural mismatch in the insertion depth of bilateral cochlear implants on speech perception

A bilateral advantage for diotically presented stimuli has been observed for cochlear implant (CI) users and is suggested to be dependent on symmetrical implant performance. Studies using CI simulations have not shown a true "bilateral" advantage, but a "better ear" effect and have demonstrated that performance decreases with increasing basalward shift in insertion depth. This study aimed to determine whether there is a bilateral advantage for CI simulations with interaurally matched insertions and the extent to which performance is affected by interaural insertion depth mismatch. Speech perception in noise and self-reported ease of listening were measured using matched bilateral, mismatched bilateral and unilateral CI simulations over four insertion depths for seventeen normal hearing listeners. Speech scores and ease of listening reduced with increasing basalward shift in (interaurally matched) insertion depth. A bilateral advantage for speech perception was only observed when the insertion depths were interaurally matched and deep. No advantage was observed for small to moderate interaural insertion-depth mismatches, consistent with a better ear effect. Finally, both measures were poorer than expected for a better ear effect for large mismatches, suggesting that misalignment of the electrode arrays may prevent a bilateral advantage and detrimentally affect perception of diotically presented speech.

Advantages from bilateral hearing in speech perception in noise with simulated cochlear implants and residual acoustic hearing

Acoustic simulations were used to study the contributions of spatial hearing that may arise from combining a cochlear implant with either a second implant or contralateral residual low-frequency acoustic hearing. Speech reception thresholds (SRTs) were measured in twenty-talker babble. Spatial separation of speech and noise was simulated using a spherical head model. While low-frequency acoustic information contralateral to the implant simulation produced substantially better SRTs there was no effect of spatial cues on SRT, even when interaural differences were artificially enhanced. Simulated bilateral implants showed a significant head shadow effect, but no binaural unmasking based on interaural time differences, and weak, inconsistent overall spatial release from masking. There was also a small but significant non-spatial summation effect. It appears that typical cochlear implant speech processing strategies may substantially reduce the utility of spatial cues, even in the absence of degraded neural processing arising from auditory deprivation.