Effects of real-time cochlear implant simulation on speech production

Discrimination and sensorimotor adaptation of self-produced vowels in cochlear implant users

Humans rely on auditory feedback to monitor and adjust their speech for clarity. Cochlear implants (CIs) have helped over a million people restore access to auditory feedback, which significantly improves speech production. However, there is substantial variability in outcomes. This study investigates the extent to which CI users can use their auditory feedback to detect self-produced sensory errors and make adjustments to their speech, given the coarse spectral resolution provided by their implants. First, we used an auditory discrimination task to assess the sensitivity of CI users to small differences in formant frequencies of their self-produced vowels. Then, CI users produced words with altered auditory feedback in order to assess sensorimotor adaptation to auditory error. Almost half of the CI users tested can detect small, within-channel differences in their self-produced vowels, and they can utilize this auditory feedback towards speech adaptation. An acoustic hearing control group showed better sensitivity to the shifts in vowels, even in CI-simulated speech, and elicited more robust speech adaptation behavior than the CI users. Nevertheless, this study confirms that CI users can compensate for sensory errors in their speech and supports the idea that sensitivity to these errors may relate to variability in production.

Speech Perception as a Function of the Number of Channels and Channel Interaction in Cochlear Implant Simulation

Objective

Speech perception relies on precise spectral and temporal cues. However, cochlear implant (CI) processing is confined to a limited frequency range, affecting the information transmitted to the auditory system. This study analyzes the influence of channel interaction and the number of channels on word recognition scores (WRS) within the CI simulation framework.

Methods

Two distinct experiments were conducted. The first experiment (n=29, average age =23 years, 14 females) evaluated the number of channels using eight, twelve, sixteen, and 22 channel vocoded and nonvocoded word lists for WRS assessment. The second experiment (n=29, average age =25 years, 16 females) explored channel interaction across low, middle, and high-interaction conditions.

Results

In the first experiment, participants scored 57.93%, 80.97%, 83.59%, 91.03%, and 95.45% under 8, 12, 16, and 22-channel vocoder and non-vocoder conditions, respectively. The number of vocoder channels significantly affected WRS, with significant differences observed in all conditions except between the 12-channel and 16-channels (p<0.01). In the second experiment, the participants scored 2.2%, 20.6%, and 50.6% under high, mid, and low interaction conditions, respectively. Statistically significant differences were observed across all channel interaction conditions (p<0.01).

Conclusions

While the number of channels had a notable impact on WRS, it is essential to note that certain conditions (12 vs. 16) did not yield statistically significant differences. The observed differences in WRS were eclipsed by the pronounced effects of channel interaction. Notably, all conditions in the channel interaction experiment exhibited statistically significant differences. These findings underscore the paramount importance of prioritizing channel interaction in signal processing and CI fitting.

Intelligibility of Word-Initial Obstruent Consonants in Mandarin-Speaking Prelingually Deafened Children With Cochlear Implants

PURPOSE

This study assessed the intelligibility of obstruent consonants in prelingually deafened Mandarin-speaking children with cochlear implants (CIs).

METHOD

Twenty-two Mandarin-speaking children with normal hearing (NH) aged 3.25-10.0 years and 35 Mandarin-speaking children with CIs aged 3.77-15.0 years were recruited to produce a list of Mandarin words composed of 17 word-initial obstruent consonants in different vowel contexts. The children with CIs were assigned to chronological age-matched (CA) and hearing age-matched (HA) subgroups with reference to the NH controls. One hundred naïve NH adult listeners were recruited for a consonant identification task that consisted of a total of 2,663 stimulus tokens through an online research platform. For each child speaker, the consonant productions were judged by seven to 12 different adult listeners. An average percentage of consonants correct was calculated across all listeners for each consonant.

RESULTS

The CI children in both the CA and HA subgroups showed lower intelligibility in their consonant productions than the NH controls. Among the 17 obstruents, both CI subgroups showed higher intelligibility for stops, but they demonstrated major problems with the sibilant fricatives and affricates and showed a different confusion pattern from the NH controls on these sibilants. Of the three places (alveolar, alveolopalatal, and retroflex) in Mandarin sibilants, both CI subgroups showed the lowest intelligibility and the greatest difficulties with alveolar sounds. For the NH children, there was a significant positive relationship between overall consonant intelligibility and chronological age. For the children with CIs, the best fit regression model revealed significant effects of chronological age and age at implantation, with their quadratic terms included.

CONCLUSIONS

Mandarin-speaking children with CIs experience major challenges in the three-way place contrasts of sibilant sounds in consonant production. Chronological age and the combined effect of CI-related time variables play important roles in the development of obstruent consonants in the CI children.

The effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production: cochlear implanted versus normal hearing individuals

Normal auditory feedback contributes to moment-to-moment control of speech production. Effects of auditory feedback's absence on hearing-impaired individuals are widely documented, but auditory perturbation has not been investigated. Our objective was to evaluate the effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production among prelingual cochlear implant (CI) users and normal hearing (NH) individuals, to evaluate CI users' reliance on auditory feedback. Twenty young adults (10 CI, 10 NH), without developmental and cognitive impairments, participated in the study. Under variable auditory feedback conditions, speech production (spontaneous or reading aloud) was measured using speech rate, percentage of interruptions, fundamental frequency (F0), and relative intensity. Results showed that (1) both DAF and FAF caused slower speech rates and more interruptions while reading aloud, with DAF having larger effect; (2) altered feedback produced no differences between groups, except an increase in F0 for CI users during DAF; and (3) CI users' ability to understand speech via phone and without lip-reading was positively correlated with performance under DAF. These findings suggest that auditory perturbation similarly affects speech production among prelingual CI users and NH individuals, indicating CI users depend on auditory feedback to the same degree as normal hearing individuals.

Supra-Segmental Changes in Speech Production as a Result of Spectral Feedback Degradation: Comparison with Lombard Speech

Perturbations to acoustic speech feedback have been typically localized to specific phonetic characteristics, for example, fundamental frequency (F0) or the first two formants (F1/F2), or affect all aspects of the speech signal equally, for example, via the addition of background noise. This paper examines the consequences of a more selective global perturbation: real-time cochlear implant (CI) simulation of acoustic speech feedback. Specifically, we examine the potential similarity between speakers' response to noise vocoding and the characteristics of Lombard speech. An acoustic analysis of supra-segmental characteristics in speaking rate, F0 production, and voice amplitude revealed changes that paralleled the Lombard effect in some domains but not others. Two studies of speech intelligibility complemented the acoustic analysis, finding that intelligibility significantly decreased as a result of CI simulation of speaker feedback. Together, the results point to differences in speakers' responses to these two superficially similar feedback manipulations. In both cases we see a complex, multi-faceted behavior on the part of talkers. We argue that more instances of global perturbation and broader response assessment are needed to determine whether such complexity is present in other feedback manipulations or if it represents a relatively rare exception to the typical compensatory feedback response.

Objective Identification of Simulated Cochlear Implant Settings in Normal-Hearing Listeners Via Auditory Cortical Evoked Potentials

OBJECTIVES

Providing cochlear implant (CI) patients the optimal signal processing settings during mapping sessions is critical for facilitating their speech perception. Here, we aimed to evaluate whether auditory cortical event-related potentials (ERPs) could be used to objectively determine optimal CI parameters.

DESIGN

While recording neuroelectric potentials, we presented a set of acoustically vocoded consonants (aKa, aSHa, and aNa) to normal-hearing listeners (n = 12) that simulated speech tokens processed through four different combinations of CI stimulation rate and number of spectral maxima. Parameter settings were selected to feature relatively fast/slow stimulation rates and high/low number of maxima; 1800 pps/20 maxima, 1800/8, 500/20 and 500/8.

RESULTS

Speech identification and reaction times did not differ with changes in either the number of maxima or stimulation rate indicating ceiling behavioral performance. Similarly, we found that conventional univariate analysis (analysis of variance) of N1 and P2 amplitude/latency failed to reveal strong modulations across CI-processed speech conditions. In contrast, multivariate discriminant analysis based on a combination of neural measures was used to create "neural confusion matrices" and identified a unique parameter set (1800/8) that maximally differentiated speech tokens at the neural level. This finding was corroborated by information transfer analysis which confirmed these settings optimally transmitted information in listeners' neural and perceptual responses.

CONCLUSIONS

Translated to actual implant patients, our findings suggest that scalp-recorded ERPs might be useful in determining optimal signal processing settings from among a closed set of parameter options and aid in the objective fitting of CI devices.