Musical pitch and lexical tone perception with cochlear implants

Prosodic and Visual Cues Facilitate Irony Comprehension by Mandarin-Speaking Children With Cochlear Implants

PURPOSE

This study investigated irony comprehension by Mandarin-speaking children with cochlear implants, focusing on how prosodic and visual cues contribute to their comprehension, and whether second-order Theory of Mind is required for using these cues.

METHOD

We tested 52 Mandarin-speaking children with cochlear implants (aged 3-7 years) and 52 age- and gender-matched children with normal hearing. All children completed a Theory of Mind test and a story comprehension test. Ironic stories were presented in three conditions, each providing different cues: (a) context-only, (b) context and prosody, and (c) context, prosody, and visual cues. Comparisons were conducted on the accuracy of story understanding across the three conditions to examine the role of prosodic and visual cues.

RESULTS

The results showed that, compared to the context-only condition, the additional prosodic and visual cues both improved the accuracy of irony comprehension for children with cochlear implants, similar to their normal-hearing peers. Furthermore, such improvements were observed for all children, regardless of whether they passed the second-order Theory of Mind test or not.

CONCLUSIONS

This study is the first to demonstrate the benefits of prosodic and visual cues in irony comprehension, without reliance on second-order Theory of Mind, for Mandarin-speaking children with cochlear implants. It implies potential insights for utilizing prosodic and visual cues in intervention strategies to promote irony comprehension.

Sustainable Benefits of High Variability Phonetic Training in Mandarin-speaking Kindergarteners With Cochlear Implants: Evidence From Categorical Perception of Lexical Tones

OBJECTIVES

Although pitch reception poses a great challenge for individuals with cochlear implants (CIs), formal auditory training (e.g., high variability phonetic training [HVPT]) has been shown to provide direct benefits in pitch-related perceptual performances such as lexical tone recognition for CI users. As lexical tones in spoken language are expressed with a multitude of distinct spectral, temporal, and intensity cues, it is important to determine the sources of training benefits for CI users. The purpose of the present study was to conduct a rigorous fine-scale evaluation with the categorical perception (CP) paradigm to control the acoustic parameters and test the efficacy and sustainability of HVPT for Mandarin-speaking pediatric CI recipients. The main hypothesis was that HVPT-induced perceptual learning would greatly enhance CI users' ability to extract the primary pitch contours from spoken words for lexical tone identification and discrimination. Furthermore, individual differences in immediate and long-term gains from training would likely be attributable to baseline performance and duration of CI use.

DESIGN

Twenty-eight prelingually deaf Mandarin-speaking kindergarteners with CIs were tested. Half of them received five sessions of HVPT within a period of 3 weeks. The other half served as control who did not receive the formal training. Two classical CP tasks on a tonal continuum from Mandarin tone 1 (high-flat in pitch) to tone 2 (mid-rising in pitch) with fixed acoustic features of duration and intensity were administered before (pretest), immediately after (posttest), and 10 weeks posttraining termination (follow-up test). Participants were instructed to either label a speech stimulus along the continuum (i.e., identification task) or determine whether a pair of stimuli separated by zero or two steps from the continuum was the same or different (i.e., discrimination task). Identification function measures (i.e., boundary position and boundary width) and discrimination function scores (i.e., between-category score, within-category score, and peakedness score) were assessed for each child participant across the three test sessions.

RESULTS

Linear mixed-effects (LME) models showed significant training-induced enhancement in lexical tone categorization with significantly narrower boundary width and better between-category discrimination in the immediate posttest over pretest for the trainees. Furthermore, training-induced gains were reliably retained in the follow-up test 10 weeks after training. By contrast, no significant changes were found in the control group across sessions. Regression analysis confirmed that baseline performance (i.e., boundary width in the pretest session) and duration of CI use were significant predictors for the magnitude of training-induced benefits.

CONCLUSIONS

The stringent CP tests with synthesized stimuli that excluded acoustic cues other than the pitch contour and were never used in training showed strong evidence for the efficacy of HVPT in yielding immediate and sustained improvement in lexical tone categorization for Mandarin-speaking children with CIs. The training results and individual differences have remarkable implications for developing personalized computer-based short-term HVPT protocols that may have sustainable long-term benefits for aural rehabilitation in this clinical population.

Musical Mistuning Perception and Appraisal in Cochlear Implant Recipients

OBJECTIVE

Music is a very crucial art form that can evoke emotions, and the harmonious presence of the human voice in music is an impactful part of this process. As a result, vocals have had some significant effects on contemporary music. The mechanism behind the cochlear implant (CI) recipients perceiving different aspects of music is clear; however, how well they perceive vocal tuning within music it is not well known. Hence, this study evaluated the mistuning perception of CI recipients and compared their performance with normal-hearing (NH) listeners.

STUDY DESIGN, SETTING, AND PATIENTS

A total of 16 CI users (7 cisgender men, 9 cisgender women) and 16 sex-matched NH controls with an average age of 30.2 (±10.9; range, 19-53) years and 23.5 (±6.1; range, 20-37) years, respectively, were enrolled in this study. We evaluated the mistuning ability using the mistuning perception test (MPT) and assessed self-perceived music perception and engagement using the music-related quality-of-life questionnaire. Test performance was measured and reported on the item-response theory metric with a z score ranging from -4 to +4.

RESULTS

A significant difference in the MPT scores was found between NH and CI recipients, whereas a significant correlation was noted between the music-related quality-of-life questionnaire-frequency subscale and MPT scores. No significant correlations were found between age, CI age, and CI usage duration and MPT performance.

CONCLUSIONS

This study revealed that musical mistuning perception is a limitation for CI recipients, similar to previously evaluated aspects of music perception. Hence, it is important to consider this aspect in the assessment of music perception, enjoyment, and music-based auditory interventions in CI recipients, as vocals are paramount in music perception and recreation. The MPT is a convenient and accessible tool for mistuning assessment in CI and hearing-aid users.

Tone perception development in Mandarin-speaking children with cochlear implants

OBJECTIVE

The purpose of this study was to examine the longer-term effects of cochlear implant (CI) use on tone perception by evaluating improvement in Mandarin tone recognition in children with CIs 2 and 3 years post CI activation, and to explore the effects of implant age, chronological age and duration of CI use on the development of tone perception.

METHODS

Tone perception was assessed in 29 bilateral profound hearing impaired children (mean chronological age = 4.6 years, SD = 0.7 years) with unilateral CIs at 24 and 36 months after CI activation using the tone perception subtest in the Mandarin Early Speech Perception (MESP-T) test.

RESULTS

Children's tone recognition for tone pairs and individual tones improved significantly between 2 and 3 years post CI use, showing an increase in average tone recognition score from 73.2% to 81.8%, which was significantly higher than chance level (i.e. 50%). There was no significant correlation between tone recognition ability and either implant age or chronological age at two evaluation points. Further analysis revealed that the tone recognition score for tone pair 2-3 was significantly lower than that for other tone pairs except for tone pair 2-4.

CONCLUSIONS

Longer CI experience can significantly improve tone recognition ability in CI children between 2 and 3 years post CI activation.

Cochlear-implant Mandarin tone recognition with a disyllabic word corpus

Despite pitch being considered the primary cue for discriminating lexical tones, there are secondary cues such as loudness contour and duration, which may allow some cochlear implant (CI) tone discrimination even with severely degraded pitch cues. To isolate pitch cues from other cues, we developed a new disyllabic word stimulus set (Di) whose primary (pitch) and secondary (loudness) cue varied independently. This Di set consists of 270 disyllabic words, each having a distinct meaning depending on the perceived tone. Thus, listeners who hear the primary pitch cue clearly may hear a different meaning from listeners who struggle with the pitch cue and must rely on the secondary loudness contour. A lexical tone recognition experiment was conducted, which compared Di with a monosyllabic set of natural recordings. Seventeen CI users and eight normal-hearing (NH) listeners took part in the experiment. Results showed that CI users had poorer pitch cues encoding and their tone recognition performance was significantly influenced by the “missing” or “confusing” secondary cues with the Di corpus. The pitch-contour-based tone recognition is still far from satisfactory for CI users compared to NH listeners, even if some appear to integrate multiple cues to achieve high scores. This disyllabic corpus could be used to examine the performance of pitch recognition of CI users and the effectiveness of pitch cue enhancement based Mandarin tone enhancement strategies. The Di corpus is freely available online: https://github.com/BetterCI/DiTone.

Pitch Accuracy of Vocal Singing in Deaf Children With Bimodal Hearing and Bilateral Cochlear Implants

OBJECTIVES

The purpose of the present study was to investigate the pitch accuracy of vocal singing in children with severe to profound hearing loss who use bilateral cochlear implants (CIs) or bimodal devices [CI at one ear and hearing aid (HA) at the other] in comparison to similarly-aged children with normal-hearing (NH).

DESIGN

The participants included four groups: (1) 26 children with NH, (2) 13 children with bimodal devices, (3) 31 children with bilateral CIs that were implanted sequentially, and (4) 10 children with bilateral CIs that were implanted simultaneously. All participants were aged between 7 and 11 years old. Each participant was recorded singing a self-chosen song that was familiar to him or her. The fundamental frequencies (F0) of individual sung notes were extracted and normalized to facilitate cross-subject comparisons. Pitch accuracy was quantified using four pitch-based metrics calculated with reference to the target music notes: mean note deviation, contour direction, mean interval deviation, and F0 variance ratio. A one-way ANOVA was used to compare listener-group difference on each pitch metric. A principal component analysis showed that the mean note deviation best accounted for pitch accuracy in vocal singing. A regression analysis examined potential predictors of CI children's singing proficiency using mean note deviation as the dependent variable and demographic and audiological factors as independent variables.

RESULTS

The results revealed significantly poorer performance on all four pitch-based metrics in the three groups of children with CIs in comparison to children with NH. No significant differences were found among the three CI groups. Among the children with CIs, variability in the vocal singing proficiency was large. Within the group of 13 bimodal users, the mean note deviation was significantly correlated with their unaided pure-tone average thresholds (r = 0.582, p = 0.037). The regression analysis for all children with CIs, however, revealed no significant demographic or audiological predictor for their vocal singing performance.

CONCLUSION

Vocal singing performance in children with bilateral CIs or bimodal devices is not significantly different from each other on a group level. Compared to children with NH, the pediatric bimodal and bilateral CI users, in general, demonstrated significant deficits in vocal singing ability. Demographic and audiological factors, known from previous studies to be associated with good speech and language development in prelingually-deafened children with CIs, were not associated with singing accuracy for these children.

Relative Weights of Temporal Envelope Cues in Different Frequency Regions for Mandarin Vowel, Consonant, and Lexical Tone Recognition

Objectives: Mandarin-speaking users of cochlear implants (CI) perform poorer than their English counterpart. This may be because present CI speech coding schemes are largely based on English. This study aims to evaluate the relative contributions of temporal envelope (E) cues to Mandarin phoneme (including vowel, and consonant) and lexical tone recognition to provide information for speech coding schemes specific to Mandarin.

Design: Eleven normal hearing subjects were studied using acoustic temporal E cues that were extracted from 30 continuous frequency bands between 80 and 7,562 Hz using the Hilbert transform and divided into five frequency regions. Percent-correct recognition scores were obtained with acoustic E cues presented in three, four, and five frequency regions and their relative weights calculated using the least-square approach.

Results: For stimuli with three, four, and five frequency regions, percent-correct scores for vowel recognition using E cues were 50.43–84.82%, 76.27–95.24%, and 96.58%, respectively; for consonant recognition 35.49–63.77%, 67.75–78.87%, and 87.87%; for lexical tone recognition 60.80–97.15%, 73.16–96.87%, and 96.73%. For frequency region 1 to frequency region 5, the mean weights in vowel recognition were 0.17, 0.31, 0.22, 0.18, and 0.12, respectively; in consonant recognition 0.10, 0.16, 0.18, 0.23, and 0.33; in lexical tone recognition 0.38, 0.18, 0.14, 0.16, and 0.14.

Conclusion: Regions that contributed most for vowel recognition was Region 2 (502–1,022 Hz) that contains first formant (F1) information; Region 5 (3,856–7,562 Hz) contributed most to consonant recognition; Region 1 (80–502 Hz) that contains fundamental frequency (F0) information contributed most to lexical tone recognition.

Pure tone discrimination with cochlear implants and filter-band spread

For many cochlear implant (CI) users, frequency discrimination is still challenging. We studied the effect of frequency differences relative to the electrode frequency bands on pure tone discrimination. A single-center, prospective, controlled, psychoacoustic exploratory study was conducted in a tertiary university referral center. Thirty-four patients with Cochlear Ltd. and MED-EL CIs and 19 age-matched normal-hearing control subjects were included. Two sinusoidal tones were presented with varying frequency differences. The reference tone frequency was chosen according to the center frequency of basal or apical electrodes. Discrimination abilities were psychophysically measured in a three-interval, two-alternative, forced-choice procedure (3I-2AFC) for various CI electrodes. Hit rates were measured, particularly with respect to discrimination abilities at the corner frequency of the electrode frequency-bands. The mean rate of correct decision concerning pitch difference was about 60% for CI users and about 90% for the normal-hearing control group. In CI users, the difference limen was two semitones, while normal-hearing participants detected the difference of one semitone. No influence of the corner frequency of the CI electrodes was found. In CI users, pure tone discrimination seems to be independent of tone positions relative to the corner frequency of the electrode frequency-band. Differences of 2 semitones can be distinguished within one electrode.

Meta-Analysis on the Identification of Linguistic and Emotional Prosody in Cochlear Implant Users and Vocoder Simulations

OBJECTIVES

This study quantitatively assesses how cochlear implants (CIs) and vocoder simulations of CIs influence the identification of linguistic and emotional prosody in nontonal languages. By means of meta-analysis, it was explored how accurately CI users and normal-hearing (NH) listeners of vocoder simulations (henceforth: simulation listeners) identify prosody compared with NH listeners of unprocessed speech (henceforth: NH listeners), whether this effect of electric hearing differs between CI users and simulation listeners, and whether the effect of electric hearing is influenced by the type of prosody that listeners identify or by the availability of specific cues in the speech signal.

DESIGN

Records were found by searching the PubMed Central, Web of Science, Scopus, Science Direct, and PsycINFO databases (January 2018) using the search terms "cochlear implant prosody" and "vocoder prosody." Records (published in English) were included that reported results of experimental studies comparing CI users' and/or simulation listeners' identification of linguistic and/or emotional prosody in nontonal languages to that of NH listeners (all ages included). Studies that met the inclusion criteria were subjected to a multilevel random-effects meta-analysis.

RESULTS

Sixty-four studies reported in 28 records were included in the meta-analysis. The analysis indicated that CI users and simulation listeners were less accurate in correctly identifying linguistic and emotional prosody compared with NH listeners, that the identification of emotional prosody was more strongly compromised by the electric hearing speech signal than linguistic prosody was, and that the low quality of transmission of fundamental frequency (f0) through the electric hearing speech signal was the main cause of compromised prosody identification in CI users and simulation listeners. Moreover, results indicated that the accuracy with which CI users and simulation listeners identified linguistic and emotional prosody was comparable, suggesting that vocoder simulations with carefully selected parameters can provide a good estimate of how prosody may be identified by CI users.

CONCLUSIONS

The meta-analysis revealed a robust negative effect of electric hearing, where CIs and vocoder simulations had a similar negative influence on the identification of linguistic and emotional prosody, which seemed mainly due to inadequate transmission of f0 cues through the degraded electric hearing speech signal of CIs and vocoder simulations.

Impact of Auditory-Motor Musical Training on Melodic Pattern Recognition in Cochlear Implant Users

OBJECTIVE

Cochlear implant (CI) users struggle with tasks of pitch-based prosody perception. Pitch pattern recognition is vital for both music comprehension and understanding the prosody of speech, which signals emotion and intent. Research in normal-hearing individuals shows that auditory-motor training, in which participants produce the auditory pattern they are learning, is more effective than passive auditory training. We investigated whether auditory-motor training of CI users improves complex sound perception, such as vocal emotion recognition and pitch pattern recognition, compared with purely auditory training.

STUDY DESIGN

Prospective cohort study.

SETTING

Tertiary academic center.

PATIENTS

Fifteen postlingually deafened adults with CIs.

INTERVENTION(S)

Participants were divided into 3 one-month training groups: auditory-motor (intervention), auditory-only (active control), and no training (control). Auditory-motor training was conducted with the "Contours" software program and auditory-only training was completed with the "AngelSound" software program.

MAIN OUTCOME MEASURE

Pre and posttest examinations included tests of speech perception (consonant-nucleus-consonant, hearing-in-noise test sentence recognition), speech prosody perception, pitch discrimination, and melodic contour identification.

RESULTS

Participants in the auditory-motor training group performed better than those in the auditory-only and no-training (p < 0.05) for the melodic contour identification task. No significant training effect was noted on tasks of speech perception, speech prosody perception, or pitch discrimination.

CONCLUSIONS

These data suggest that short-term auditory-motor music training of CI users impacts pitch pattern recognition. This study offers approaches for enriching the world of complex sound in the CI user.

Bimodal benefits in Mandarin-speaking cochlear implant users for music perception and tone recognition

BACKGROUND

Cochlear implant (CI) users have difficulty appreciating music and perceiving lexical tones in Mandarin Chinese. Wearing a hearing aid (HA) in the contralateral ear for bimodal hearing may provide additional benefits.

OBJECTIVES

To measure the bimodal benefits of music perception and tone recognition and to investigate the relationship between the two in Mandarin-speaking bimodal CI subjects.

MATERIALS AND METHODS

Sixteen Mandarin-speaking bimodal CI subjects (aged between 16 and 49 years) participated in the study. Music perception (pitch discrimination, melody discrimination and instrument identification) and lexical tone recognition were tested with electric stimulation (CI alone) or bimodal stimulation (CI + HA).

RESULTS

Subjects showed a significant bimodal benefit in tone recognition in quiet and noise, and in all music perception tests. The bimodal benefit for tone recognition in noise was significantly correlated with that of pitch discrimination thresholds and instrument identification scores.

CONCLUSION

Mandarin-speaking bimodal CI users achieved better music perception and tone recognition ability with CI + HA than with CI alone. The bimodal benefit of tone recognition was significantly correlated with that of music perception.

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants

OBJECTIVES

Normally-hearing (NH) listeners rely more on prosodic cues than on lexical-semantic cues for emotion perception in speech. In everyday spoken communication, the ability to decipher conflicting information between prosodic and lexical-semantic cues to emotion can be important: for example, in identifying sarcasm or irony. Speech degradation in cochlear implants (CIs) can be sufficiently overcome to identify lexical-semantic cues, but the distortion of voice pitch cues makes it particularly challenging to hear prosody with CIs. The purpose of this study was to examine changes in relative reliance on prosodic and lexical-semantic cues in NH adults listening to spectrally degraded speech and adult CI users. We hypothesized that, compared with NH counterparts, CI users would show increased reliance on lexical-semantic cues and reduced reliance on prosodic cues for emotion perception. We predicted that NH listeners would show a similar pattern when listening to CI-simulated versions of emotional speech.

DESIGN

Sixteen NH adults and 8 postlingually deafened adult CI users participated in the study. Sentences were created to convey five lexical-semantic emotions (angry, happy, neutral, sad, and scared), with five sentences expressing each category of emotion. Each of these 25 sentences was then recorded with the 5 (angry, happy, neutral, sad, and scared) prosodic emotions by 2 adult female talkers. The resulting stimulus set included 125 recordings (25 Sentences × 5 Prosodic Emotions) per talker, of which 25 were congruent (consistent lexical-semantic and prosodic cues to emotion) and the remaining 100 were incongruent (conflicting lexical-semantic and prosodic cues to emotion). The recordings were processed to have 3 levels of spectral degradation: full-spectrum, CI-simulated (noise-vocoded) to have 8 channels and 16 channels of spectral information, respectively. Twenty-five recordings (one sentence per lexical-semantic emotion recorded in all five prosodies) were used for a practice run in the full-spectrum condition. The remaining 100 recordings were used as test stimuli. For each talker and condition of spectral degradation, listeners indicated the emotion associated with each recording in a single-interval, five-alternative forced-choice task. The responses were scored as proportion correct, where "correct" responses corresponded to the lexical-semantic emotion. CI users heard only the full-spectrum condition.

RESULTS

The results showed a significant interaction between hearing status (NH, CI) and congruency in identifying the lexical-semantic emotion associated with the stimuli. This interaction was as predicted, that is, CI users showed increased reliance on lexical-semantic cues in the incongruent conditions, while NH listeners showed increased reliance on the prosodic cues in the incongruent conditions. As predicted, NH listeners showed increased reliance on lexical-semantic cues to emotion when the stimuli were spectrally degraded.

CONCLUSIONS

The present study confirmed previous findings of prosodic dominance for emotion perception by NH listeners in the full-spectrum condition. Further, novel findings with CI patients and NH listeners in the CI-simulated conditions showed reduced reliance on prosodic cues and increased reliance on lexical-semantic cues to emotion. These results have implications for CI listeners' ability to perceive conflicts between prosodic and lexical-semantic cues, with repercussions for their identification of sarcasm and humor. Understanding instances of sarcasm or humor can impact a person's ability to develop relationships, follow conversation, understand vocal emotion and intended message of a speaker, following jokes, and everyday communication in general.

Effects of Fundamental Frequency Contours on Sentence Recognition in Mandarin-Speaking Children With Cochlear Implants

Purpose Fundamental frequency (F0) is the primary acoustic cue for lexical tone perception in tonal languages but is processed in a limited way in cochlear implant (CI) systems. The aim of this study was to evaluate the importance of F0 contours in sentence recognition in Mandarin-speaking children with CIs and find out whether it is similar to/different from that in age-matched normal-hearing (NH) peers. Method Age-appropriate sentences, with F0 contours manipulated to be either natural or flattened, were randomly presented to preschool children with CIs and their age-matched peers with NH under three test conditions: in quiet, in white noise, and with competing sentences at 0 dB signal-to-noise ratio. Results The neutralization of F0 contours resulted in a significant reduction in sentence recognition. While this was seen only in noise conditions among NH children, it was observed throughout all test conditions among children with CIs. Moreover, the F0 contour-induced accuracy reduction ratios (i.e., the reduction in sentence recognition resulting from the neutralization of F0 contours compared to the normal F0 condition) were significantly greater in children with CIs than in NH children in all test conditions. Conclusions F0 contours play a major role in sentence recognition in both quiet and noise among pediatric implantees, and the contribution of the F0 contour is even more salient than that in age-matched NH children. These results also suggest that there may be differences between children with CIs and NH children in how F0 contours are processed.

Music perception and training for pediatric cochlear implant users

INTRODUCTION

Cochlear implants (CIs) are biomedical devices that restore sound perception for people with severe-to-profound sensorineural hearing loss. Most postlingually deafened CI users are able to achieve excellent speech recognition in quiet environments. However, current CI sound processors remain limited in their ability to deliver fine spectrotemporal information, making it difficult for CI users to perceive complex sounds. Limited access to complex acoustic cues such as music, environmental sounds, lexical tones, and voice emotion may have significant ramifications on quality of life, social development, and community interactions.

AREAS COVERED

The purpose of this review article is to summarize the literature on CIs and music perception, with an emphasis on music training in pediatric CI recipients. The findings have implications on our understanding of noninvasive, accessible methods for improving auditory processing and may help advance our ability to improve sound quality and performance for implantees.

EXPERT OPINION

Music training, particularly in the pediatric population, may be able to continue to enhance auditory processing even after performance plateaus. The effects of these training programs appear generalizable to non-trained musical tasks, speech prosody and, emotion perception. Future studies should employ rigorous control groups involving a non-musical acoustic intervention, standardized auditory stimuli, and the provision of feedback.

A Cross-Language Comparison of Sentence Recognition Using American English and Mandarin Chinese HINT and AzBio Sentences

OBJECTIVES

The aim of this study was to perform a cross-language comparison of two commonly used sentence-recognition materials (i.e., Hearing in Noise Test [HINT] and AzBio) in American English (AE) and Mandarin Chinese (MC).

DESIGNS

Sixty normal-hearing, native English-speaking and 60 normal-hearing, native Chinese-speaking young adults were recruited to participate in three experiments. In each experiment, the subjects were tested in their native language. In experiments I and II, noise and tone vocoders were used to process the HINT and AzBio sentences, respectively. The number of channels varied from 1 to 9, with an envelope cutoff frequency of 160 Hz. In experiment III, the AE AzBio and the MC HINT sentences were tested in speech-shaped noise at various signal to noise ratios (i.e., -20, -15, -10, -5, and 0 dB). The performance-intensity functions of sentence recognition using the two sets of sentence materials were compared.

RESULTS

Results of experiments I and II using vocoder processing indicated that the AE and MC versions of HINT and AzBio sentences differed in level of difficulty. The AE version yielded higher recognition performance than the MC version for both HINT and AzBio sentences. The type of vocoder processing (i.e., tone and noise vocoders) produced little differences in sentence-recognition performance in both languages. Incidentally, the AE AzBio sentences and the MC HINT sentences had similar recognition performance under vocoder processing. Such similarity was further confirmed under noise conditions in experiment III, where the performance-intensity functions of the two sets of sentences were closely matched.

CONCLUSIONS

The HINT and AzBio sentence materials developed in AE and MC differ in level of difficulty. The AE AzBio and the MC HINT sentence materials are similar in level of difficulty. In cross-language comparative research, the MC HINT and the AE AzBio sentences should be chosen for the respective language as the target sentence-recognition test materials.

A Neurophysiological Study of Musical Pitch Identification in Mandarin-Speaking Cochlear Implant Users

Music perception in cochlear implant (CI) users is far from satisfactory, not only because of the technological limitations of current CI devices but also due to the neurophysiological alterations that generally accompany deafness. Early behavioral studies revealed that similar mechanisms underlie musical and lexical pitch perception in CI-based electric hearing. Although neurophysiological studies of the musical pitch perception of English-speaking CI users are actively ongoing, little such research has been conducted with Mandarin-speaking CI users; as Mandarin is a tonal language, these individuals require pitch information to understand speech. The aim of this work was to study the neurophysiological mechanisms accounting for the musical pitch identification abilities of Mandarin-speaking CI users and normal-hearing (NH) listeners. Behavioral and mismatch negativity (MMN) data were analyzed to examine musical pitch processing performance. Moreover, neurophysiological results from CI users with good and bad pitch discrimination performance (according to the just-noticeable differences (JND) and pitch-direction discrimination (PDD) tasks) were compared to identify cortical responses associated with musical pitch perception differences. The MMN experiment was conducted using a passive oddball paradigm, with musical tone C4 (262 Hz) presented as the standard and tones D4 (294 Hz), E4 (330 Hz), G#4 (415 Hz), and C5 (523 Hz) presented as deviants. CI users demonstrated worse musical pitch discrimination ability than did NH listeners, as reflected by larger JND and PDD thresholds for pitch identification, and significantly increased latencies and reduced amplitudes in MMN responses. Good CI performers had better MMN results than did bad performers. Consistent with findings for English-speaking CI users, the results of this work suggest that MMN is a viable marker of cortical pitch perception in Mandarin-speaking CI users.

Mandarin lexical tone recognition in bimodal cochlear implant users

Objective: To assess the recognition of lexical tones in Mandarin-speaking bimodal cochlear implant (CI) subjects.Design: Lexical tone recognition in quiet and noise (SNR= +5 dB) was measured with electric stimulation (CI alone) or bimodal stimulation (CI + hearing aid (HA)). The recognition and confusion rates of the four tones (T1, T2, T3 and T4) were analysed. Spearman correlation analysis was performed to examine the relationship between hearing levels in the contralateral ear and bimodal benefits.Study sample: Twenty native Mandarin-speaking bimodal CI users, with ages ranging from 16-49 years.Results: Relative to the CI alone, mean tone recognition with the CI + HA improved significantly from 84.1-92.1% correct in quiet (+8 points) and from 57.9-73.1% correct in noise (+15.2 points). Tone confusions between T2 and T3 were the most prominent in all test conditions, and T4 tended to be labelled as T3 in noise. There was no significant correlation between the bimodal benefits for tone recognition and the unaided or HA-aided pure-tone thresholds at 0.25 kHz.Conclusion: Listeners with CI + HA exhibited significantly better tone recognition than with CI alone. The bimodal advantage for tone recognition was greater in noise than in quiet, perhaps due to a ceiling effect in quiet.

Musical Emotion Perception in Bimodal Patients: Relative Weighting of Musical Mode and Tempo Cues

Several cues are used to convey musical emotion, the two primary being musical mode and musical tempo. Specifically, major and minor modes tend to be associated with positive and negative valence, respectively, and songs at fast tempi have been associated with more positive valence compared to songs at slow tempi (Balkwill and Thompson, 1999; Webster and Weir, 2005). In Experiment I, we examined the relative weighting of musical tempo and musical mode among adult cochlear implant (CI) users combining electric and contralateral acoustic stimulation, or "bimodal" hearing. Our primary hypothesis was that bimodal listeners would utilize both tempo and mode cues in their musical emotion judgments in a manner similar to normal-hearing listeners. Our secondary hypothesis was that low-frequency (LF) spectral resolution in the non-implanted ear, as quantified via psychophysical tuning curves (PTCs) at 262 and 440 Hz, would be significantly correlated with degree of bimodal benefit for musical emotion perception. In Experiment II, we investigated across-channel spectral resolution using a spectral modulation detection (SMD) task and neural representation of temporal fine structure via the frequency following response (FFR) for a 170-ms /da/ stimulus. Results indicate that CI-alone performance was driven almost exclusively by tempo cues, whereas bimodal listening demonstrated use of both tempo and mode. Additionally, bimodal benefit for musical emotion perception may be correlated with spectral resolution in the non-implanted ear via SMD, as well as neural representation of F0 amplitude via FFR - though further study with a larger sample size is warranted. Thus, contralateral acoustic hearing can offer significant benefit for musical emotion perception, and the degree of benefit may be dependent upon spectral resolution of the non-implanted ear.

On the Relationship Between General Auditory Sensitivity and Speech Perception: An Examination of Pitch and Lexical Tone Perception in 4- to 6-Year-Old Children

Purpose Theoretical models and substantial research have proposed that general auditory sensitivity is a developmental foundation for speech perception and language acquisition. Nonetheless, controversies exist about the effectiveness of general auditory training in improving speech and language skills. This research investigated the relationships among general auditory sensitivity, phonemic speech perception, and word-level speech perception via the examination of pitch and lexical tone perception in children. Method Forty-eight typically developing 4- to 6-year-old Cantonese-speaking children were tested on the discrimination of the pitch patterns of lexical tones in synthetic stimuli, discrimination of naturally produced lexical tones, and identification of lexical tone in familiar words. Results The findings revealed that accurate lexical tone discrimination and identification did not necessarily entail the accurate discrimination of nonlinguistic stimuli that followed the pitch levels and pitch shapes of lexical tones. Although pitch discrimination and tone discrimination abilities were strongly correlated, accuracy in pitch discrimination was lower than that in tone discrimination, and nonspeech pitch discrimination ability did not precede linguistic tone discrimination in the developmental trajectory. Conclusions Contradicting the theoretical models, the findings of this study suggest that general auditory sensitivity and speech perception may not be causally or hierarchically related. The finding that accuracy in pitch discrimination is lower than that in tone discrimination suggests that comparable nonlinguistic auditory perceptual ability may not be necessary for accurate speech perception and language learning. The results cast doubt on the use of nonlinguistic auditory perceptual training to improve children's speech, language, and literacy abilities.

Age-sensitive associations of segmental and suprasegmental perception with sentence-level language skills in Mandarin-speaking children with cochlear implants

BACKGROUND AND AIM

It remains unclear how recognition of segmental and suprasegmental phonemes contributes to sentence-level language processing skills in Mandarin-speaking children with cochlear implants (CIs). Our study examined the influence of implantation age on the recognition of consonants, lexical tones and sentences respectively, and more importantly, the contribution of phonological skills to sentence repetition accuracy in Mandarin-speaking children with CIs.

METHODS

The participants were three groups of prelingually deaf children who received cochlear implants at various ages and their age-matched controls with normal hearing. Three tasks were administered to assess their consonant perception, lexical tone recognition and language skills in open-set sentence repetition.

RESULTS

Children with CIs lagged behind NH peers in all the three tests, and performances on segmental, suprasegmental and sentence-level processing were differentially modulated by implantation age. Furthermore, performances on recognition of consonants and lexical tones were significant predictors of sentence repetition accuracy in the children with CIs.

CONCLUSION

Overall, segmental and suprasegmental perception as well as sentence-level processing is impaired in Mandarin-speaking children with CIs compared with age-matched children with NH. In children with CIs recognition of segmental and suprasegmental phonemes at the lower level predicts sentence repetition accuracy at the higher level. More importantly, implantation age plays an important role in the development of phonological skills and higher-order language skills, suggesting that age-appropriate aural rehabilitation and speech intervention programs need to be developed in order to better help CI users who receive CIs at different ages.

Processing of Acoustic Information in Lexical Tone Production and Perception by Pediatric Cochlear Implant Recipients

Purpose: This study examined the utilization of multiple types of acoustic information in lexical tone production and perception by pediatric cochlear implant (CI) recipients who are native speakers of Mandarin Chinese. Methods: Lexical tones were recorded from CI recipients and their peers with normal hearing (NH). Each participant was asked to produce a disyllabic word, yan jing, with which the first syllable was pronounced as Tone 3 (a low dipping tone) while the second syllable was pronounced as Tone 1 (a high level tone, meaning "eyes") or as Tone 4 (a high falling tone, meaning "eyeglasses"). In addition, a parametric manipulation in fundamental frequency (F0) and duration of Tones 1 and 4 used in a lexical tone recognition task in Peng et al. (2017) was adopted to evaluate the perceptual reliance on each dimension. Results: Mixed-effect analyses of duration, intensity, and F0 cues revealed that NH children focused exclusively on marking distinct F0 contours, while CI participants shortened Tone 4 or prolonged Tone 1 to enhance their contrast. In line with these production strategies, NH children relied primarily on F0 cues to identify the two tones, whereas CI children showed greater reliance on duration cues. Moreover, CI participants who placed greater perceptual weight on duration cues also tended to exhibit smaller changes in their F0 production. Conclusion: Pediatric CI recipients appear to contrast the secondary acoustic dimension (duration) in addition to F0 contours for both lexical tone production and perception. These findings suggest that perception and production strategies of lexical tones are well coupled in this pediatric CI population.

Singing Proficiency of Members of a Choir Formed by Prelingually Deafened Children With Cochlear Implants

Purpose A group of 10 prelingually deafened children with cochlear implants (CIs) formed a choir and received 21 months of formal music training. The purpose of this study was to evaluate the singing proficiency of these children. Method The participants included all choir members (7 girls and 3 boys, mean age of 9.5 years old) who were unilateral CI users. Meanwhile, 8 age-matched children with normal hearing were recruited as controls and were trained on 1 song for 2 weeks. Individual singing samples without instrument accompaniment were recorded from all participants. The singing samples were subject to acoustic analysis in which the fundamental frequency (F0) of each note was extracted and the duration was measured. Five metrics were developed and computed to quantify the accuracy of their pitch and rhythm performance. The 5 metrics included (a) percent correct of F0 contour direction of adjacent notes, (b) mean deviation of the normalized F0 across the notes, (c) mean deviation of the pitch intervals, (d) mean deviation of adjacent note duration ratio, and (e) mean absolute deviation of note duration. Results The choir members with CIs demonstrated high accuracy in both pitch and tempo measures and performed on par with the children with normal hearing. Early start of music training after implantation and use of bimodal hearing contributed to the development of better music ability in these children with CIs. Conclusion These findings indicated that rigorous music training could facilitate high singing proficiency in prelingually deafened children with CIs.

Music Training Can Improve Music and Speech Perception in Pediatric Mandarin-Speaking Cochlear Implant Users

Due to limited spectral resolution, cochlear implants (CIs) do not convey pitch information very well. Pitch cues are important for perception of music and tonal language; it is possible that music training may improve performance in both listening tasks. In this study, we investigated music training outcomes in terms of perception of music, lexical tones, and sentences in 22 young (4.8 to 9.3 years old), prelingually deaf Mandarin-speaking CI users. Music perception was measured using a melodic contour identification (MCI) task. Speech perception was measured for lexical tones and sentences presented in quiet. Subjects received 8 weeks of MCI training using pitch ranges not used for testing. Music and speech perception were measured at 2, 4, and 8 weeks after training was begun; follow-up measures were made 4 weeks after training was stopped. Mean baseline performance was 33.2%, 76.9%, and 45.8% correct for MCI, lexical tone recognition, and sentence recognition, respectively. After 8 weeks of MCI training, mean performance significantly improved by 22.9, 14.4, and 14.5 percentage points for MCI, lexical tone recognition, and sentence recognition, respectively (p < .05 in all cases). Four weeks after training was stopped, there was no significant change in posttraining music and speech performance. The results suggest that music training can significantly improve pediatric Mandarin-speaking CI users’ music and speech perception.

The Advances in Hearing Rehabilitation and Cochlear Implants in China

Hearing loss (HL) is a common sensory impairment in humans, with significant economic and social impacts. With nearly 20% of the world's population, China has focused on economic development and health awareness to improve the care for its hearing-impaired population. Recently, the Chinese government has initiated national programs such as the China Disabled Persons Federation to fund prevention, treatment, and rehabilitation of hearing impairment. Newborn hearing screening and auditory rehabilitation programs in China have expanded exponentially with government support. While facing many challenges and overcoming obstacles, cochlear implantation (CI) programs in China have also experienced considerable growth. This review discusses the implementation of CI programs for HL in China and presents current HL data including epidemiology, newborn hearing screening, and determination of genetic etiologies. Sharing the experience in Chinese auditory rehabilitation and CI programs will shine a light on the developmental pathway of healthcare infrastructure to meet emerging needs of the hearing-impaired population in other developing countries.

A Follow-Up Study on Music and Lexical Tone Perception in Adult Mandarin-Speaking Cochlear Implant Users

OBJECTIVE

The aim was to evaluate the development of music and lexical tone perception in Mandarin-speaking adult cochlear implant (CI) users over a period of 1 year.

STUDY DESIGN

Prospective patient series.

SETTING

Tertiary hospital and research institute.

PATIENTS

Twenty five adult CI users, with ages ranging from 19 to 75 years old, participated in a year-long follow-up evaluation. There were also 40 normal hearing adult subjects who participated as a control group to provide the normal value range.

INTERVENTIONS

Musical sounds in cochlear implants (Mu.S.I.C.) test battery was undertaken to evaluate music perception ability. Mandarin Tone Identification in Noise Test (M-TINT) was used to assess lexical tone recognition. The tests for CI users were completed at 1, 3, 6, and 12 months after the CI switch-on.

MAIN OUTCOMES MEASURES

Quantitative and statistical analysis of their results from music and tone perception tests.

RESULTS

The performance of music perception and tone recognition both demonstrated an overall improvement in outcomes during the entire 1-year follow-up process. The increasing trends were obvious in the early period especially in the first 6 months after switch-on. There was a significant improvement in the melody discrimination (p < 0.01), timbre identification (p < 0.001), tone recognition in quiet (p < 0.0001), and in noise (p < 0.0001).

CONCLUSIONS

Adult Mandarin-speaking CI users show an increasingly improved performance on music and tone perception during the 1-year follow-up. The improvement was the most prominent in the first 6 months of CI use. It is essential to strengthen the rehabilitation training within the first 6 months.

Processing of Acoustic Cues in Lexical-Tone Identification by Pediatric Cochlear-Implant Recipients

PURPOSE

The objective was to investigate acoustic cue processing in lexical-tone recognition by pediatric cochlear-implant (CI) recipients who are native Mandarin speakers.

METHOD

Lexical-tone recognition was assessed in pediatric CI recipients and listeners with normal hearing (NH) in 2 tasks. In Task 1, participants identified naturally uttered words that were contrastive in lexical tones. For Task 2, a disyllabic word (yanjing) was manipulated orthogonally, varying in fundamental-frequency (F0) contours and duration patterns. Participants identified each token with the second syllable jing pronounced with Tone 1 (a high level tone) as eyes or with Tone 4 (a high falling tone) as eyeglasses.

RESULTS

CI participants' recognition accuracy was significantly lower than NH listeners' in Task 1. In Task 2, CI participants' reliance on F0 contours was significantly less than that of NH listeners; their reliance on duration patterns, however, was significantly higher than that of NH listeners. Both CI and NH listeners' performance in Task 1 was significantly correlated with their reliance on F0 contours in Task 2.

CONCLUSION

For pediatric CI recipients, lexical-tone recognition using naturally uttered words is primarily related to their reliance on F0 contours, although duration patterns may be used as an additional cue.

Relative contributions of acoustic temporal fine structure and envelope cues for lexical tone perception in noise

Previous studies have shown that lexical tone perception in quiet relies on the acoustic temporal fine structure (TFS) but not on the envelope (E) cues. The contributions of TFS to speech recognition in noise are under debate. In the present study, Mandarin tone tokens were mixed with speech-shaped noise (SSN) or two-talker babble (TTB) at five signal-to-noise ratios (SNRs; -18 to +6 dB). The TFS and E were then extracted from each of the 30 bands using Hilbert transform. Twenty-five combinations of TFS and E from the sound mixtures of the same tone tokens at various SNRs were created. Twenty normal-hearing, native-Mandarin-speaking listeners participated in the tone-recognition test. Results showed that tone-recognition performance improved as the SNRs in either TFS or E increased. The masking effects on tone perception for the TTB were weaker than those for the SSN. For both types of masker, the perceptual weights of TFS and E in tone perception in noise was nearly equivalent, with E playing a slightly greater role than TFS. Thus, the relative contributions of TFS and E cues to lexical tone perception in noise or in competing-talker maskers differ from those in quiet and those to speech perception of non-tonal languages.

Speech recognition outcomes in Mandarin-speaking cochlear implant users with fine structure processing

CONCLUSION

Mandarin-speaking adults can use the Fine Structure Processing (FSP) coding strategy as well as the Continuous Interleaved Sampling (CIS+) coding strategy. No loss in performance was observed after switch-over. Tone identification improves over time with the FSP coding strategy, which is of benefit to tonal-language users. After some time, fine structure was preferred.

OBJECTIVE

This study aimed to determine speech perception, tone perception, and the subjective preferences of Mandarin-speaking adults who received the FSP coding strategy, at upgrade from the CIS + coding strategy.

METHODS

Thirteen Mandarin-speaking subjects were tested at switch-over from CIS + to the FSP coding strategy ∼1-month after switch-over, 2-months after switch-over, and 3-months after switch-over with the Mandarin Hearing in Noise Test (M-HINT), the Mandarin Tone Identification in Noise Test (M-TINT), and a visual analogue scale assessing Sound and Speech Assessment (SSA).

RESULTS

There were no significant differences in the M-HINT between presentation levels (62 dB SPL vs 65 dB SPL), over time, nor when compared to the CIS + coding strategy. Tone perception improved significantly over time with the FSP coding strategy. Subjects rated the FSP coding strategy with the OPUS 2 as significantly more 'full' and 'rich' than with the CIS + coding strategy after 3-months.

Voice emotion perception and production in cochlear implant users

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Mandarin Tone Identification in Cochlear Implant Users Using Exaggerated Pitch Contours

OBJECTIVE

To determine whether exaggerating the variations in fundamental frequency (F0) contours of Mandarin-based pitch fluctuations could improve tone identification by cochlear implant (CI) users.

METHODS

Twelve normal-hearing (NH) listeners and 11 CI users were tested for their ability to recognize F0 contours modeled after Mandarin tones, in 4- or 5-alternatives forced-choice paradigms. Two types of stimuli were used: computer-generated complex tones and voice recordings. Four contours were tested with voice recordings: flat, rise, fall, and dip. A fifth contour, peak, was added for complex tones. The F0 range of each contour was varied in an adaptive manner. A maximum-likelihood technique was used to fit a psychometric function to the performance data and extract threshold at 70% accuracy.

RESULTS

As F0 range increased, performance in tone identification improved but did not reach 100% for some CI users, suggesting that confusions between contours could always be made even with extremely exaggerated contours. Compared with NH participants, CI users required substantially larger F0 ranges to identify tones, on the order of 9.3 versus 0.4 semitones. CI users achieved better performance for complex tones than for voice recordings, whereas the reverse was true for NH participants. Confusion matrices showed that the "flat" tone was often a default option when the tone contour's F0 range presented was too narrow for participants to respond correctly.

CONCLUSION

These results demonstrate markedly impaired ability for CI users to identify tonal contours, but suggest that the use of exaggerated pitch contours may be helpful for tonal language perception.

Relationship between Intelligence Quotient and Musical Ability in Children with Cochlear Implantation

INTRODUCTION

Children with cochlear implants (CIs) may experience few opportunities for positive musical experiences, and musical perception is therefore often not sufficiently developed. This paper investigates and discusses the relationship between intelligence quotient (IQ) and musical ability in children with CIs compared with children with normal hearing.

MATERIALS AND METHODS

This was a comparative analytical study conducted in 48 children with unilateral CI and 48 normal-hearing children, 6-8 years of age, with 'normal' IQ and no formal music training. The average IQ score in the experimental and control groups were 105.41 and 106.31, respectively. No statistically significant differences were detected between Raven's IQ scores in both groups. Data were collected by administering Raven's Colored Progressive Matrices IQ Tests and the Montreal Battery of Evaluation of Musical Abilities (MBEMA) Test, consisting of scale, contour, interval, rhythm, and memory sections.

RESULTS

Mean total MBEMA score in the experimental and control groups was 58.93 and 72.16 (out of 100), respectively. Significant differences were evident between scores of children with CIs in comparison with their normal-hearing peers (P≤0.001). A remarkable direct correlation between IQ and musical scores in both the control (r≥0.38) and experimental (r≥0.37) groups was observed.

CONCLUSION

IQ has a noticeable effect on music processing and facilitates the perception of various musical elements. With regard to the mutual relationship between IQ and musical skills, this study illustrates the advantage of determining music perception scores and highlights the importance of appropriate musical intervention in order to enhance auditory neural plasticity, especially in children with cochlear implantation.

Pitch and lexical tone perception of bilingual English-Mandarin-speaking cochlear implant recipients, hearing aid users, and normally hearing listeners

OBJECTIVES

The purpose of this current study was to investigate whether pitch, lexical tone, and/or speech-in-noise perception were significantly correlated for Singaporean teenagers or adults who spoke both Mandarin and English.

METHODS

Thirty-three normal hearing or near-normal hearing listeners who did not use a hearing device (NNH group), eight postlingually deafened cochlear implant (CI) recipients (CI group), and three postlingually deafened bilateral hearing aid (HA) users (HA group) were recruited. All participants were bilingual Mandarin-English-speaking Singaporean residents. Participants were assessed on tests of pitch-ranking, lexical tone perception, and speech-in-noise.

RESULTS

The NNH group scored significantly better than the CI group for all tests and subtests. There were no significant differences for the pitch test between the HA group and either the CI or NNH group. However, HA users scored significantly better than the CI group, and more aligned with the NNH group's scores for both the lexical tone and Mandarin speech-in-noise test. There were highly significant moderate positive correlations between all three tests. Discussion Overall, the performance of the CI users in this study indicates that CI recipients still struggle on pitch-related auditory perception tasks. Additionally, although the test scores from the HA users were better than the CI recipients, they were not as good as the NNH listeners. The significant moderate correlations between all three tests indicate that there is at least some degree of overlap in the skills required to accurately perceive these stimuli.

CONCLUSION

The overall results suggest that CI users, and to a lesser extent HA users, still struggle with complex auditory perceptual tasks, particularly when it requires the perception of pitch. However, it may be possible that training one of these skills (e.g. musical pitch) may then generalize to other tasks (e.g. lexical tone and/or speech-in-noise). This is important for counseling, as well as for planning effective rehabilitation programs.

Lexical tone recognition in noise in normal-hearing children and prelingually deafened children with cochlear implants

OBJECTIVE

The purpose of the present study was to investigate Mandarin tone recognition in background noise in children with cochlear implants (CIs), and to examine the potential factors contributing to their performance.

DESIGN

Tone recognition was tested using a two-alternative forced-choice paradigm in various signal-to-noise ratio (SNR) conditions (i.e. quiet, +12, +6, 0, and -6 dB). Linear correlation analysis was performed to examine possible relationships between the tone-recognition performance of the CI children and the demographic factors.

STUDY SAMPLE

Sixty-six prelingually deafened children with CIs and 52 normal-hearing (NH) children as controls participated in the study.

RESULTS

Children with CIs showed an overall poorer tone-recognition performance and were more susceptible to noise than their NH peers. Tone confusions between Mandarin tone 2 and tone 3 were most prominent in both CI and NH children except for in the poorest SNR conditions. Age at implantation was significantly correlated with tone-recognition performance of the CI children in noise.

CONCLUSIONS

There is a marked deficit in tone recognition in prelingually deafened children with CIs, particularly in noise listening conditions. While factors that contribute to the large individual differences are still elusive, early implantation could be beneficial to tone development in pediatric CI users.

Loudness Contour Can Influence Mandarin Tone Recognition: Vocoder Simulation and Cochlear Implants

Lexical tone recognition with current cochlear implants (CI) remains unsatisfactory due to significantly degraded pitch-related acoustic cues, which dominate the tone recognition by normal-hearing (NH) listeners. Several secondary cues (e.g., amplitude contour, duration, and spectral envelope) that influence tone recognition in NH listeners and CI users have been studied. This work proposes a loudness contour manipulation algorithm, namely Loudness-Tone (L-Tone), to investigate the effects of loudness contour on Mandarin tone recognition and the effectiveness of using loudness cue to enhance tone recognition for CI users. With L-Tone, the intensity of sound samples is multiplied by gain values determined by instantaneous fundamental frequencies (F0s) and pre-defined gain- F0 mapping functions. Perceptual experiments were conducted with a four-channel noise-band vocoder simulation in NH listeners and with CI users. The results suggested that 1) loudness contour is a useful secondary cue for Mandarin tone recognition, especially when pitch cues are significantly degraded; 2) L-Tone can be used to improve Mandarin tone recognition in both simulated and actual CI-hearing without significant negative effect on vowel and consonant recognition. L-Tone is a promising algorithm for incorporation into real-time CI processing and off-line CI rehabilitation training software.

Results using the OPAL strategy in Mandarin speaking cochlear implant recipients

OBJECTIVE

To evaluate the effectiveness of an experimental pitch-coding strategy for improving recognition of Mandarin lexical tone in cochlear implant (CI) recipients.

DESIGN

Adult CI recipients were tested on recognition of Mandarin tones in quiet and speech-shaped noise at a signal-to-noise ratio of +10 dB; Mandarin sentence speech-reception threshold (SRT) in speech-shaped noise; and pitch discrimination of synthetic complex-harmonic tones in quiet. Two versions of the experimental strategy were examined: (OPAL) linear (1:1) mapping of fundamental frequency (F0) to the coded modulation rate; and (OPAL+) transposed mapping of high F0s to a lower coded rate. Outcomes were compared to results using the clinical ACE™ strategy.

STUDY SAMPLE

Five Mandarin speaking users of Nucleus® cochlear implants.

RESULTS

A small but significant benefit in recognition of lexical tones was observed using OPAL compared to ACE in noise, but not in quiet, and not for OPAL+ compared to ACE or OPAL in quiet or noise. Sentence SRTs were significantly better using OPAL+ and comparable using OPAL to those using ACE. No differences in pitch discrimination thresholds were observed across strategies.

CONCLUSIONS

OPAL can provide benefits to Mandarin lexical tone recognition in moderately noisy conditions and preserve perception of Mandarin sentences in challenging noise conditions.

Music-based training for pediatric CI recipients: A systematic analysis of published studies

In recent years, there has been growing interest in the use of music-based training to enhance speech and language development in children with normal hearing and some forms of communication disorders, including pediatric CI users. The use of music training for CI users may initially seem incongruous given that signal processing for CIs presents a degraded version of pitch and timbre, both key elements in music. Furthermore, empirical data of systematic studies of music training, particularly in relation to transfer to speech skills are limited. This study describes the rationale for music training of CI users, describes key features of published studies of music training with CI users, and highlights some developmental and logistical issues that should be taken into account when interpreting or planning studies of music training and speech outcomes with pediatric CI recipients.

Melodic Contour Training and Its Effect on Speech in Noise, Consonant Discrimination, and Prosody Perception for Cochlear Implant Recipients

Cochlear implant (CI) recipients generally have good perception of speech in quiet environments but difficulty perceiving speech in noisy conditions, reduced sensitivity to speech prosody, and difficulty appreciating music. Auditory training has been proposed as a method of improving speech perception for CI recipients, and recent efforts have focussed on the potential benefits of music-based training. This study evaluated two melodic contour training programs and their relative efficacy as measured on a number of speech perception tasks. These melodic contours were simple 5-note sequences formed into 9 contour patterns, such as "rising" or "rising-falling." One training program controlled difficulty by manipulating interval sizes, the other by note durations. Sixteen adult CI recipients (aged 26-86 years) and twelve normal hearing (NH) adult listeners (aged 21-42 years) were tested on a speech perception battery at baseline and then after 6 weeks of melodic contour training. Results indicated that there were some benefits for speech perception tasks for CI recipients after melodic contour training. Specifically, consonant perception in quiet and question/statement prosody was improved. In comparison, NH listeners performed at ceiling for these tasks. There was no significant difference between the posttraining results for either training program, suggesting that both conferred benefits for training CI recipients to better perceive speech.

Melodic pitch perception and lexical tone perception in Mandarin-speaking cochlear implant users

OBJECTIVES

To examine the relationship between lexical tone perception and melodic pitch perception in Mandarin-speaking cochlear implant (CI) users and to investigate the influence of previous acoustic hearing on CI users' speech and music perception.

DESIGN

Lexical tone perception and melodic contour identification (MCI) were measured in 21 prelingual and 11 postlingual young (aged 6-26 years) Mandarin-speaking CI users. Lexical tone recognition was measured for four tonal patterns: tone 1 (flat F0), tone 2 (rising F0), tone 3 (falling-rising F0), and tone 4 (falling F0). MCI was measured using nine five-note melodic patterns that contained changes in pitch contour, as well as different semitone spacing between notes.

RESULTS

Lexical tone recognition was generally good (overall mean = 81% correct), and there was no significant difference between subject groups. MCI performance was generally poor (mean = 23% correct). MCI performance was significantly better for postlingual (mean = 32% correct) than for prelingual CI participants (mean = 18% correct). After correcting for outliers, there was no significant correlation between lexical tone recognition and MCI performance for prelingual or postlingual CI participants. Age at deafness was significantly correlated with MCI performance only for postlingual participants. CI experience was significantly correlated with MCI performance for both prelingual and postlingual participants. Duration of deafness was significantly correlated with tone recognition only for prelingual participants.

CONCLUSIONS

Despite the prevalence of pitch cues in Mandarin, the present CI participants had great difficulty perceiving melodic pitch. The availability of amplitude and duration cues in lexical tones most likely compensated for the poor pitch perception observed with these CI listeners. Previous acoustic hearing experience seemed to benefit postlingual CI users' melodic pitch perception. Longer CI experience was associated with better MCI performance for both subject groups, suggesting that CI users' music perception may improve as they gain experience with their device.

Benefits of music training in mandarin-speaking pediatric cochlear implant users

PURPOSE

The aims of this study were to assess young (5- to 10-year-old) Mandarin-speaking cochlear implant (CI) users' musical pitch perception and to assess the benefits of computer-based home training on performance.

METHOD

Melodic contour identification (MCI) was used to assess musical pitch perception in 14 Mandarin-speaking pediatric CI users; the instrument timbre and the contour length were varied as experimental parameters. Six subjects received subsequent MCI training on their home computer in which auditory and visual feedback were provided.

RESULTS

MCI performance was generally poor (grand mean=33.3% correct) and highly variable, with scores ranging from 9.3% to 98.1% correct; there was no significant effect of instrument timbre or contour length on performance (p>.05). After 4 weeks of training, performance sharply improved. Follow-up measures that were conducted 8 weeks after training was stopped showed no significant decline in MCI performance. For the 6 trained subjects, there was a significant effect of contour length for the training and follow-up measures.

CONCLUSION

These preliminary data suggest that although baseline MCI performance initially may be poor, training may greatly improve Mandarin-speaking pediatric CI users' melodic pitch perception.

Stimulating on multiple electrodes can improve temporal pitch perception

OBJECTIVE

The objective of this study was to test if stimulating multiple electrodes can improve temporal pitch ranking performance at low and high stimulation rates.

DESIGN

Temporal pitch cues are usually based on modifying the stimulation rate of the implant and thereby provide a continuum of pitches on a single electrode up to approximately 300 Hz.

STUDY SAMPLE

Ten cochlear implant subjects were asked to pitch rank stimuli presented with direct electrical stimulation. The pulses were applied on one, three, six, or eleven electrodes. In one of the conditions the current amplitude of each pulse was randomly varied between 0 and 100%. Their frequency ranged from 100 up to 500 pps.

RESULTS

Listeners showed the previously reported performance pattern in most conditions with very good performance at the lowest standard rates and deteriorating performance to near chance level at the highest rate tested. Performance with eleven electrodes was significantly better than performance with one electrode at 500 pps.

CONCLUSION

Stimulating on multiple electrodes can improve temporal pitch perception.

Mandarin speech perception in combined electric and acoustic stimulation

For deaf individuals with residual low-frequency acoustic hearing, combined use of a cochlear implant (CI) and hearing aid (HA) typically provides better speech understanding than with either device alone. Because of coarse spectral resolution, CIs do not provide fundamental frequency (F0) information that contributes to understanding of tonal languages such as Mandarin Chinese. The HA can provide good representation of F0 and, depending on the range of aided acoustic hearing, first and second formant (F1 and F2) information. In this study, Mandarin tone, vowel, and consonant recognition in quiet and noise was measured in 12 adult Mandarin-speaking bimodal listeners with the CI-only and with the CI+HA. Tone recognition was significantly better with the CI+HA in noise, but not in quiet. Vowel recognition was significantly better with the CI+HA in quiet, but not in noise. There was no significant difference in consonant recognition between the CI-only and the CI+HA in quiet or in noise. There was a wide range in bimodal benefit, with improvements often greater than 20 percentage points in some tests and conditions. The bimodal benefit was compared to CI subjects’ HA-aided pure-tone average (PTA) thresholds between 250 and 2000 Hz; subjects were divided into two groups: “better” PTA (<50 dB HL) or “poorer” PTA (>50 dB HL). The bimodal benefit differed significantly between groups only for consonant recognition. The bimodal benefit for tone recognition in quiet was significantly correlated with CI experience, suggesting that bimodal CI users learn to better combine low-frequency spectro-temporal information from acoustic hearing with temporal envelope information from electric hearing. Given the small number of subjects in this study (n = 12), further research with Chinese bimodal listeners may provide more information regarding the contribution of acoustic and electric hearing to tonal language perception.

Perception of polyphony with cochlear implants for 2 and 3 simultaneous pitches

HYPOTHESIS

It was hypothesized that cochlear implant (CI) subjects would be able to correctly identify 1, 2, and 3 simultaneous pitches through direct electrical stimulation. We further hypothesized that the location on the implant array and the fundamental frequency of the pitches would have an impact on the performance.

BACKGROUND

"They gave me back speech but not music" is a sentence commonly heard by CI subjects. One of the reasons is that in music, multiple streams are frequently played at the same time, which is an essential feature of harmony. Current CI speech processors do not allow CI users to perceive such complex polyphonic sounds.

METHODS

In the present study, the authors assessed the ability of CI subjects to perceive simultaneous modulation frequencies based on direct electrical stimulation. Ten CI subjects were asked to identify 1, 2, and 3 simultaneous pitches applied on different electrodes using sinusoidal amplitude modulation. All stimuli were loudness balanced before the actual identification task.

RESULTS

Subjects were able to identify 1, 2, and 3 simultaneous pitches. The further the distance between the 2 electrodes, the better was the performance in the 2-pitch condition. The distance between the modulation frequencies had a significant effect on the performance in the 2-and 3-pitch condition.

CONCLUSION

Subjects are able to identify complex polyphonic stimuli based on the number of active electrodes. The additional polyphonic rate pitch cue improves performance in some conditions.

Melodic interval perception by normal-hearing listeners and cochlear implant users

The perception of melodic intervals (sequential pitch differences) is essential to music perception. This study tested melodic interval perception in normal-hearing (NH) listeners and cochlear implant (CI) users. Melodic interval ranking was tested using an adaptive procedure. CI users had slightly higher interval ranking thresholds than NH listeners. Both groups' interval ranking thresholds, although not affected by root note, significantly increased with standard interval size and were higher for descending intervals than for ascending intervals. The pitch direction effect may be due to a procedural artifact or a difference in central processing. In another test, familiar melodies were played with all the intervals scaled by a single factor. Subjects rated how in tune the melodies were and adjusted the scaling factor until the melodies sounded the most in tune. CI users had lower final interval ratings and less change in interval rating as a function of scaling factor than NH listeners. For CI users, the root-mean-square error of the final scaling factors and the width of the interval rating function were significantly correlated with the average ranking threshold for ascending rather than descending intervals, suggesting that CI users may have focused on ascending intervals when rating and adjusting the melodies.

Experimental assessment of polyphonic tones with cochlear implants

HYPOTHESIS

We hypothesized that cochlear implant (CI) users are able to discriminate tones consisting of 1 and 2 modulation frequencies when the stimuli are applied through direct electrical stimulation.

BACKGROUND

Music perception is a very challenging task for CI users. In music, multiple tones often occur simultaneously, an essential feature of harmony. Proper encoding of simultaneous tones is crucial to musical perception and appreciation. With current implant processing strategies, CI users are severely impaired in the perception of pitch and polyphony.

METHODS

The ability of CI users to identify the number of simultaneous tones was assessed. Stimuli were applied with direct electrical stimulation. Stimuli with 1 modulation frequency were applied on a basal, a middle, and an apical electrode to determine if there was an effect of cochlear region. Stimuli with 2 modulation frequencies were applied on combinations of an apical electrode together with a basal or a middle electrode. Additionally, 2 modulations frequencies were presented at the same time on an apical electrode only.

RESULTS

Results demonstrate that subjects were generally able to identify the number of modulation frequencies in the presented stimuli. Performance for 1 modulation frequency stimuli was significantly above chance level on all 3 electrodes tested. Performance was best on the apical and the middle electrode, followed by the basal electrode. Subjects were also able to identify 2 modulation frequencies significantly above chance level on all 3 combinations tested. Performance was best on combination apical-basal followed by apical-middle. Performance was worst when 2 modulation frequencies were applied on an apical electrode only, but it was still significantly above chance level.

CONCLUSION

If sound processing strategies were to use concurrent modulation frequencies on multiple or single electrodes, then possibly polyphonic tones would be better perceived by CI users yielding better music and language perception.

Mandarin speech perception in combined electric and acoustic stimulation

For deaf individuals with residual low-frequency acoustic hearing, combined use of a cochlear implant (CI) and hearing aid (HA) typically provides better speech understanding than with either device alone. Because of coarse spectral resolution, CIs do not provide fundamental frequency (F0) information that contributes to understanding of tonal languages such as Mandarin Chinese. The HA can provide good representation of F0 and, depending on the range of aided acoustic hearing, first and second formant (F1 and F2) information. In this study, Mandarin tone, vowel, and consonant recognition in quiet and noise was measured in 12 adult Mandarin-speaking bimodal listeners with the CI-only and with the CI+HA. Tone recognition was significantly better with the CI+HA in noise, but not in quiet. Vowel recognition was significantly better with the CI+HA in quiet, but not in noise. There was no significant difference in consonant recognition between the CI-only and the CI+HA in quiet or in noise. There was a wide range in bimodal benefit, with improvements often greater than 20 percentage points in some tests and conditions. The bimodal benefit was compared to CI subjects' HA-aided pure-tone average (PTA) thresholds between 250 and 2000 Hz; subjects were divided into two groups: "better" PTA (<50 dB HL) or "poorer" PTA (>50 dB HL). The bimodal benefit differed significantly between groups only for consonant recognition. The bimodal benefit for tone recognition in quiet was significantly correlated with CI experience, suggesting that bimodal CI users learn to better combine low-frequency spectro-temporal information from acoustic hearing with temporal envelope information from electric hearing. Given the small number of subjects in this study (n = 12), further research with Chinese bimodal listeners may provide more information regarding the contribution of acoustic and electric hearing to tonal language perception.

Acoustic properties of vocal singing in prelingually-deafened children with cochlear implants or hearing aids

OBJECTIVE

The purpose of the present study was to investigate vocal singing performance of hearing-impaired children with cochlear implants (CI) and hearing aids (HA) as well as to evaluate the relationship between demographic factors of those hearing-impaired children and their singing ability.

METHODS

Thirty-seven prelingually-deafened children with CIs and 31 prelingually-deafened children with HAs, and 37 normal-hearing (NH) children participated in the study. The fundamental frequencies (F0) of each note in the recorded songs were extracted and the duration of each sung note was measured. Five metrics were used to evaluate the pitch-related and rhythm-based aspects of singing accuracy.

RESULTS

Children with CIs and HAs showed significantly poorer performance in either the pitch-based assessments or the rhythm-based measure than the NH children. No significant differences were seen between the CI and HA groups in all of these measures except for the mean deviation of the pitch intervals. For both hearing-impaired groups, length of device use was significantly correlated with singing accuracy.

CONCLUSIONS

There is a marked deficit in vocal singing ability either in pitch or rhythm accuracy in a majority of prelingually-deafened children who have received CIs or fitted with HAs. Although an increased length of device use might facilitate singing performance to some extent, the chance for the hearing-impaired children fitted with either HAs or CIs to reach high proficiency in singing is quite slim.