Mandarin tone recognition in cochlear-implant subjects

Prelingually Deaf Children With Cochlear Implants Show Better Perception of Voice Cues and Speech in Competing Speech Than Postlingually Deaf Adults With Cochlear Implants

OBJECTIVES

Postlingually deaf adults with cochlear implants (CIs) have difficulties with perceiving differences in speakers' voice characteristics and benefit little from voice differences for the perception of speech in competing speech. However, not much is known yet about the perception and use of voice characteristics in prelingually deaf implanted children with CIs. Unlike CI adults, most CI children became deaf during the acquisition of language. Extensive neuroplastic changes during childhood could make CI children better at using the available acoustic cues than CI adults, or the lack of exposure to a normal acoustic speech signal could make it more difficult for them to learn which acoustic cues they should attend to. This study aimed to examine to what degree CI children can perceive voice cues and benefit from voice differences for perceiving speech in competing speech, comparing their abilities to those of normal-hearing (NH) children and CI adults.

DESIGN

CI children's voice cue discrimination (experiment 1), voice gender categorization (experiment 2), and benefit from target-masker voice differences for perceiving speech in competing speech (experiment 3) were examined in three experiments. The main focus was on the perception of mean fundamental frequency (F0) and vocal-tract length (VTL), the primary acoustic cues related to speakers' anatomy and perceived voice characteristics, such as voice gender.

RESULTS

CI children's F0 and VTL discrimination thresholds indicated lower sensitivity to differences compared with their NH-age-equivalent peers, but their mean discrimination thresholds of 5.92 semitones (st) for F0 and 4.10 st for VTL indicated higher sensitivity than postlingually deaf CI adults with mean thresholds of 9.19 st for F0 and 7.19 st for VTL. Furthermore, CI children's perceptual weighting of F0 and VTL cues for voice gender categorization closely resembled that of their NH-age-equivalent peers, in contrast with CI adults. Finally, CI children had more difficulties in perceiving speech in competing speech than their NH-age-equivalent peers, but they performed better than CI adults. Unlike CI adults, CI children showed a benefit from target-masker voice differences in F0 and VTL, similar to NH children.

CONCLUSION

Although CI children's F0 and VTL voice discrimination scores were overall lower than those of NH children, their weighting of F0 and VTL cues for voice gender categorization and their benefit from target-masker differences in F0 and VTL resembled that of NH children. Together, these results suggest that prelingually deaf implanted CI children can effectively utilize spectrotemporally degraded F0 and VTL cues for voice and speech perception, generally outperforming postlingually deaf CI adults in comparable tasks. These findings underscore the presence of F0 and VTL cues in the CI signal to a certain degree and suggest other factors contributing to the perception challenges faced by CI adults.

Galvanic vs. pulsatile effects on decision-making networks: reshaping the neural activation landscape

Objective. Primarily due to safety concerns, biphasic pulsatile stimulation (PS) is the present standard for electrical excitation of neural tissue with a diverse set of applications. While pulses have been shown to be effective to achieve functional outcomes, they have well-known deficits. Due to recent technical advances, galvanic stimulation (GS), delivery of current for extended periods of time (>1 s), has re-emerged as an alternative to PS.Approach. In this paper, we use a winner-take-all decision-making cortical network model to investigate differences between pulsatile and GS in the context of a perceptual decision-making task.Main results. Based on previous work, we hypothesized that GS would produce more spatiotemporally distributed, network-sensitive neural responses, while PS would produce highly synchronized activation of a limited group of neurons. Our results in-silico support these hypotheses for low-amplitude GS but deviate when galvanic amplitudes are large enough to directly activate or block nearby neurons.Significance. We conclude that with careful parametrization, GS could overcome some limitations of PS to deliver more naturalistic firing patterns in the group of targeted neurons.

Cochlear-implant Mandarin tone recognition with a disyllabic word corpus

Despite pitch being considered the primary cue for discriminating lexical tones, there are secondary cues such as loudness contour and duration, which may allow some cochlear implant (CI) tone discrimination even with severely degraded pitch cues. To isolate pitch cues from other cues, we developed a new disyllabic word stimulus set (Di) whose primary (pitch) and secondary (loudness) cue varied independently. This Di set consists of 270 disyllabic words, each having a distinct meaning depending on the perceived tone. Thus, listeners who hear the primary pitch cue clearly may hear a different meaning from listeners who struggle with the pitch cue and must rely on the secondary loudness contour. A lexical tone recognition experiment was conducted, which compared Di with a monosyllabic set of natural recordings. Seventeen CI users and eight normal-hearing (NH) listeners took part in the experiment. Results showed that CI users had poorer pitch cues encoding and their tone recognition performance was significantly influenced by the “missing” or “confusing” secondary cues with the Di corpus. The pitch-contour-based tone recognition is still far from satisfactory for CI users compared to NH listeners, even if some appear to integrate multiple cues to achieve high scores. This disyllabic corpus could be used to examine the performance of pitch recognition of CI users and the effectiveness of pitch cue enhancement based Mandarin tone enhancement strategies. The Di corpus is freely available online: https://github.com/BetterCI/DiTone.

The effect of lexical tone experience on English intonation perception in Mandarin-speaking cochlear-implanted children

To examine the effect of lexical tone experience on English intonation perception in Mandarin-speaking cochlear-implanted children during second language acquisition in Taiwan. A retrospective cohort study. A tertiary referred center. Fourteen children with cochlear implant (CI) in the experimental group, and 9 normal hearing children in the control group were enrolled in this study. Cochlear implantation and hearing rehabilitation. Two speech recognition accuracies were examined: (1) Lexical tone recognition (4-alternative forced choice, AFC), (2) English Sentence Intonation (2AFC). The overall accuracies for tone perception are 61.13% (standard deviation, SD = 10.84%) for CI group and 93.82% (SD = 1.80%) for normal hearing group. Tone 4 and Tone 1 were more easily to be recognized than tone 2 and tone 3 in the pediatric CI recipients (cCI) group. In English intonation perception, the overall accuracies are 61.82% (SD = 16.85%) for CI group, and 97.59% (SD = 4.73%) for normal hearing group. Significant high correlation (R = .919, P ≦ .000) between lexical tone perception and English intonation perception is noted. There is no significant difference for English intonation perception accuracies between Mandarin-speaking cCI (61.82%) and English-speaking cCI (70.13%, P = .11). Mandarin-speaking cochlear-implanted children showed significant deficits in perception of lexical tone and English intonation relative to normal hearing children. There was no tonal language benefit in Mandarin-speaking cochlear-implanted children's English intonation perception, compared to the English-speaking cochlear-implanted peers. For cochlear-implanted children, better lexical tone perception comes with better English intonation perception. Enhancing Mandarin prosodic perception for cochlear-implanted children may benefit their command of intonation in English.

Difference in Network Effects of Pulsatile and Galvanic Stimulation

Biphasic pulsatile stimulation is the present standard for neural prosthetic use, and it is used to understand connectivity and functionality of the brain in brain mapping studies. While pulses have been shown to drive behavioral changes, such as biasing decision making, they have deficits. For example, cochlear implants restore hearing but lack the ability to restore pitch perception. Recent work shows that pulses produce artificial synchrony in networks of neurons and non-linear changes in firing rate with pulse amplitude. Studies also show galvanic stimulation, delivery of current for extended periods of time, produces more naturalistic behavioral responses than pulses. In this paper, we use a winner-take-all decision-making network model to investigate differences between pulsatile and galvanic stimulation at the single neuron and network level while accurately modeling the effects of pulses on neurons for the first time. Results show pulses bias spike timing and make neurons more resistive to natural network inputs than galvanic stimulation at an equivalent current amplitude. Clinical Relevance- This establishes that pulsatile stimulation may disrupt natural spike timing and network-level interactions while certain parameterizations of galvanic stimulation avoid these effects and can drive network firing more naturally.

The Effectiveness of Unilateral Cochlear Implantation on Performance-Based and Patient-Reported Outcome Measures in Finnish Recipients

Understanding speech is essential for adequate social interaction, and its functioning affects health, wellbeing, and quality of life (QoL). Untreated hearing loss (HL) is associated with reduced social activity, depression and cognitive decline. Severe and profound HL is routinely rehabilitated with cochlear implantation. The success of treatment is mostly assessed by performance-based outcome measures such as speech perception. The ultimate goal of cochlear implantation, however, is to improve the patient's QoL. Therefore, patient-reported outcomes measures (PROMs) would be clinically valuable as they assess subjective benefits and overall effectiveness of treatment. The aim of this study was to assess the patient-reported benefits of unilateral cochlear implantation in an unselected Finnish patient cohort of patients with bilateral HL. The study design was a prospective evaluation of 118 patients. The patient cohort was longitudinally followed up with repeated within-subject measurements preoperatively and at 6 and 12 months postoperatively. The main outcome measures were one performance-based speech-in-noise (SiN) test (Finnish Matrix Sentence Test), and two PROMs [Finnish versions of the Speech, Spatial, Qualities of Hearing questionnaire (SSQ) and the Nijmegen Cochlear Implant Questionnaire (NCIQ)]. The results showed significant average improvements in SiN scores, from +0.8 dB signal-to-noise ratio (SNR) preoperatively to -3.7 and -3.8 dB SNR at 6 and12 month follow-up, respectively. Significant improvements were also found for SSQ and NCIQ scores in all subdomains from the preoperative state to 6 and 12 months after first fitting. No clinically significant improvements were observed in any of the outcome measures between 6 and 12 months. Preoperatively, poor SiN scores were associated with low scoring in several subdomains of the SSQ and NCIQ. Poor preoperative SiN scores and low PROMs scoring were significantly associated with larger postoperative improvements. No significant association was found between SiN scores and PROMs postoperatively. This study demonstrates significant benefits of cochlear implantation in the performance-based and patient-reported outcomes in an unselected patient sample. The lack of association between performance and PROMs scores postoperatively suggests that both capture unique aspects of benefit, highlighting the need to clinically implement PROMs in addition to performance-based measures for a more holistic assessment of treatment benefit.

Acoustic analysis of tone production in Mandarin-speaking bimodal cochlear implant users

The benefit of using a hearing aid with a cochlear implant (bimodal hearing) has been demonstrated for tone perception under certain conditions. The present study evaluated bimodal effects for tone production by comparing performance between a bimodal and a unimodal implant group. Results showed that acoustic differentiation of tones produced by the bimodal group was better than the unimodal implant group, and performance was dependent on the subject's acoustic thresholds but not related to implant experience or age at implantation. The findings support the use of amplified acoustic hearing in conjunction with the implant for better development of pitch production.

Just-Noticeable Differences of Fundamental Frequency Change in Mandarin-Speaking Children with Cochlear Implants

Fundamental frequency (F0) provides the primary acoustic cue for lexical tone perception in tonal languages but remains poorly represented in cochlear implant (CI) systems. Currently, there is still a lack of understanding of sensitivity to F0 change in CI users who speak tonal languages. In the present study, just-noticeable differences (JNDs) of F0 contour and F0 level changes in Mandarin-speaking children with CIs were measured and compared with those in their age-matched normal-hearing (NH) peers. Results showed that children with CIs demonstrated significantly larger JND of F0 contour (JND-C) change and F0 level (JND-L) change compared to NH children. Further within-group comparison revealed that the JND-C change was significantly smaller than the JND-L change among children with CIs, whereas the opposite pattern was observed among NH children. No significant correlations were seen between JND-C change/JND-L change and age at implantation /duration of CI use. The contrast between children with CIs and NH children in sensitivity to F0 contour and F0 level change suggests different mechanisms of F0 processing in these two groups as a result of different hearing experiences.

Relative Weights of Temporal Envelope Cues in Different Frequency Regions for Mandarin Vowel, Consonant, and Lexical Tone Recognition

Objectives: Mandarin-speaking users of cochlear implants (CI) perform poorer than their English counterpart. This may be because present CI speech coding schemes are largely based on English. This study aims to evaluate the relative contributions of temporal envelope (E) cues to Mandarin phoneme (including vowel, and consonant) and lexical tone recognition to provide information for speech coding schemes specific to Mandarin.

Design: Eleven normal hearing subjects were studied using acoustic temporal E cues that were extracted from 30 continuous frequency bands between 80 and 7,562 Hz using the Hilbert transform and divided into five frequency regions. Percent-correct recognition scores were obtained with acoustic E cues presented in three, four, and five frequency regions and their relative weights calculated using the least-square approach.

Results: For stimuli with three, four, and five frequency regions, percent-correct scores for vowel recognition using E cues were 50.43–84.82%, 76.27–95.24%, and 96.58%, respectively; for consonant recognition 35.49–63.77%, 67.75–78.87%, and 87.87%; for lexical tone recognition 60.80–97.15%, 73.16–96.87%, and 96.73%. For frequency region 1 to frequency region 5, the mean weights in vowel recognition were 0.17, 0.31, 0.22, 0.18, and 0.12, respectively; in consonant recognition 0.10, 0.16, 0.18, 0.23, and 0.33; in lexical tone recognition 0.38, 0.18, 0.14, 0.16, and 0.14.

Conclusion: Regions that contributed most for vowel recognition was Region 2 (502–1,022 Hz) that contains first formant (F1) information; Region 5 (3,856–7,562 Hz) contributed most to consonant recognition; Region 1 (80–502 Hz) that contains fundamental frequency (F0) information contributed most to lexical tone recognition.

A Machine Learning-based Neural Implant Front End for Inducing Naturalistic Firing

Despite being able to restore speech perception with 99% success rate, cochlear implants cannot successfully restore pitch perception or music appreciation. Studies suggest that if auditory neurons were activated with fine timing closer to that of natural responses pitch would be restored. Predicting the timing of cochlear responses requires detailed biophysical models of sound transmission, inner hair cell responses, and outer hair cell responses. Performing these calculations is computationally costly for real time cochlear implant stimulation. Instead, implants typically modulate pulse amplitude of fixed pulse rate stimulation with the band-limited envelopes of incoming sound. This method is known to produce unrealistic responses, even to simple step inputs. Here we investigate using a machine learning algorithm to optimize the prediction of the desired firing patterns of the auditory afferents in response to sinusoidal and step modulation of pure tones. We conclude that a trained network that consists of 25 GRU nodes can reproduce fine timing with 4.4 percent error on a test set of sines and steps. This trained network can also transfer learn and capture features of natural sounds that are not captured by standard CI algorithms. Additionally, for 0.5 second test inputs, the ML algorithm completed the sound to spike rate conversion in 300x less time than the phenomenological model. This calculation occurs at a real-time compatible rate of 1 ms for 1 second of spike timing prediction on an i9 microprocessor. This suggests that this is a feasible approach to pursue for real-time CI implementation.

The Relative Weight of Temporal Envelope Cues in Different Frequency Regions for Mandarin Disyllabic Word Recognition

Objectives

Acoustic temporal envelope (E) cues containing speech information are distributed across all frequency spectra. To provide a theoretical basis for the signal coding of hearing devices, we examined the relative weight of E cues in different frequency regions for Mandarin disyllabic word recognition in quiet.

Design

E cues were extracted from 30 continuous frequency bands within the range of 80 to 7,562 Hz using Hilbert decomposition and assigned to five frequency regions from low to high. Disyllabic word recognition of 20 normal-hearing participants were obtained using the E cues available in two, three, or four frequency regions. The relative weights of the five frequency regions were calculated using least-squares approach.

Results

Participants correctly identified 3.13-38.13%, 27.50-83.13%, or 75.00-93.13% of words when presented with two, three, or four frequency regions, respectively. Increasing the number of frequency region combinations improved recognition scores and decreased the magnitude of the differences in scores between combinations. This suggested a synergistic effect among E cues from different frequency regions. The mean weights of E cues of frequency regions 1-5 were 0.31, 0.19, 0.26, 0.22, and 0.02, respectively.

Conclusion

For Mandarin disyllabic words, E cues of frequency regions 1 (80-502 Hz) and 3 (1,022-1,913 Hz) contributed more to word recognition than other regions, while frequency region 5 (3,856-7,562) contributed little.

Voice Emotion Recognition by Mandarin-Speaking Children with Cochlear Implants

Objectives

Emotional expressions are very important in social interactions. Children with cochlear implants can have voice emotion recognition deficits due to device limitations. Mandarin-speaking children with cochlear implants may face greater challenges than those speaking nontonal languages; the pitch information is not well preserved in cochlear implants, and such children could benefit from child-directed speech, which carries more exaggerated distinctive acoustic cues for different emotions. This study investigated voice emotion recognition, using both adult-directed and child-directed materials, in Mandarin-speaking children with cochlear implants compared with normal hearing peers. The authors hypothesized that both the children with cochlear implants and those with normal hearing would perform better with child-directed materials than with adult-directed materials.

Design

Thirty children (7.17-17 years of age) with cochlear implants and 27 children with normal hearing (6.92-17.08 years of age) were recruited in this study. Participants completed a nonverbal reasoning test, speech recognition tests, and a voice emotion recognition task. Children with cochlear implants over the age of 10 years also completed the Chinese version of the Nijmegen Cochlear Implant Questionnaire to evaluate the health-related quality of life. The voice emotion recognition task was a five-alternative, forced-choice paradigm, which contains sentences spoken with five emotions (happy, angry, sad, scared, and neutral) in a child-directed or adult-directed manner.

Results

Acoustic analyses showed substantial variations across emotions in all materials, mainly on measures of mean fundamental frequency and fundamental frequency range. Mandarin-speaking children with cochlear implants displayed a significantly poorer performance than normal hearing peers in voice emotion perception tasks, regardless of whether the performance is measured in accuracy scores, Hu value, or reaction time. Children with cochlear implants and children with normal hearing were mainly affected by the mean fundamental frequency in speech emotion recognition tasks. Chronological age had a significant effect on speech emotion recognition in children with normal hearing; however, there was no significant correlation between chronological age and accuracy scores in speech emotion recognition in children with implants. Significant effects of specific emotion and test materials (better performance with child-directed materials) in both groups of children were observed. Among the children with cochlear implants, age at implantation, percentage scores of nonverbal intelligence quotient test, and sentence recognition threshold in quiet could predict recognition performance in both accuracy scores and Hu values. Time wearing cochlear implant could predict reaction time in emotion perception tasks among children with cochlear implants. No correlation was observed between the accuracy score in voice emotion perception and the self-reported scores of health-related quality of life; however, the latter were significantly correlated with speech recognition skills among Mandarin-speaking children with cochlear implants.

Conclusions

Mandarin-speaking children with cochlear implants could have significant deficits in voice emotion recognition tasks compared with their normally hearing peers and can benefit from the exaggerated prosody of child-directed speech. The effects of age at cochlear implantation, speech and language development, and cognition could play an important role in voice emotion perception by Mandarin-speaking children with cochlear implants.

Perception of Child-Directed Versus Adult-Directed Emotional Speech in Pediatric Cochlear Implant Users

OBJECTIVES

Cochlear implants (CIs) are remarkable in allowing individuals with severe to profound hearing loss to perceive speech. Despite these gains in speech understanding, however, CI users often struggle to perceive elements such as vocal emotion and prosody, as CIs are unable to transmit the spectro-temporal detail needed to decode affective cues. This issue becomes particularly important for children with CIs, but little is known about their emotional development. In a previous study, pediatric CI users showed deficits in voice emotion recognition with child-directed stimuli featuring exaggerated prosody. However, the large intersubject variability and differential developmental trajectory known in this population incited us to question the extent to which exaggerated prosody would facilitate performance in this task. Thus, the authors revisited the question with both adult-directed and child-directed stimuli.

DESIGN

Vocal emotion recognition was measured using both child-directed (CDS) and adult-directed (ADS) speech conditions. Pediatric CI users, aged 7-19 years old, with no cognitive or visual impairments and who communicated through oral communication with English as the primary language participated in the experiment (n = 27). Stimuli comprised 12 sentences selected from the HINT database. The sentences were spoken by male and female talkers in a CDS or ADS manner, in each of the five target emotions (happy, sad, neutral, scared, and angry). The chosen sentences were semantically emotion-neutral. Percent correct emotion recognition scores were analyzed for each participant in each condition (CDS vs. ADS). Children also completed cognitive tests of nonverbal IQ and receptive vocabulary, while parents completed questionnaires of CI and hearing history. It was predicted that the reduced prosodic variations found in the ADS condition would result in lower vocal emotion recognition scores compared with the CDS condition. Moreover, it was hypothesized that cognitive factors, perceptual sensitivity to complex pitch changes, and elements of each child's hearing history may serve as predictors of performance on vocal emotion recognition.

RESULTS

Consistent with our hypothesis, pediatric CI users scored higher on CDS compared with ADS speech stimuli, suggesting that speaking with an exaggerated prosody-akin to "motherese"-may be a viable way to convey emotional content. Significant talker effects were also observed in that higher scores were found for the female talker for both conditions. Multiple regression analysis showed that nonverbal IQ was a significant predictor of CDS emotion recognition scores while Years using CI was a significant predictor of ADS scores. Confusion matrix analyses revealed a dependence of results on specific emotions; for the CDS condition's female talker, participants had high sensitivity (d' scores) to happy and low sensitivity to the neutral sentences while for the ADS condition, low sensitivity was found for the scared sentences.

CONCLUSIONS

In general, participants had higher vocal emotion recognition to the CDS condition which also had more variability in pitch and intensity and thus more exaggerated prosody, in comparison to the ADS condition. Results suggest that pediatric CI users struggle with vocal emotion perception in general, particularly to adult-directed speech. The authors believe these results have broad implications for understanding how CI users perceive emotions both from an auditory communication standpoint and a socio-developmental perspective.

Meta-Analytic Findings on Reading in Children With Cochlear Implants

This meta-analysis study aims to quantify the group differences in reading skills between children with cochlear implants and their hearing peers and between children with cochlear implants and children with hearing aids (aged between 3 and 18 years old). Of the 5,642 articles screened, 47 articles met predetermined inclusion criteria (published between 2002 and 2019). The robust variance estimation based meta-analysis models were used to synthesize all the effect sizes. Children with cochlear implants scored significantly lower than their hearing peers in phonological awareness (g = -1.62, p < 0.001), vocabulary (g = -1.50, p < 0.001), decoding (g = -1.24, p < 0.001), and reading comprehension (g = -1.39, p < 0.001), but not for fluency (g = -0.67, p = 0.054). Compared to children with hearing aids, children with cochlear implants scored significantly lower in phonological awareness (g = -0.30, p = 0.028). The percentage of unilateral cochlear implant negatively impacts the group difference between children with cochlear implants and their hearing peers. Findings from this study confirm a positive shift in reading outcomes for profoundly deaf children due to cochlear implantation. Some children with cochlear implants may need additional supports in educational settings.

Effects of Fundamental Frequency Contours on Sentence Recognition in Mandarin-Speaking Children With Cochlear Implants

Purpose Fundamental frequency (F0) is the primary acoustic cue for lexical tone perception in tonal languages but is processed in a limited way in cochlear implant (CI) systems. The aim of this study was to evaluate the importance of F0 contours in sentence recognition in Mandarin-speaking children with CIs and find out whether it is similar to/different from that in age-matched normal-hearing (NH) peers. Method Age-appropriate sentences, with F0 contours manipulated to be either natural or flattened, were randomly presented to preschool children with CIs and their age-matched peers with NH under three test conditions: in quiet, in white noise, and with competing sentences at 0 dB signal-to-noise ratio. Results The neutralization of F0 contours resulted in a significant reduction in sentence recognition. While this was seen only in noise conditions among NH children, it was observed throughout all test conditions among children with CIs. Moreover, the F0 contour-induced accuracy reduction ratios (i.e., the reduction in sentence recognition resulting from the neutralization of F0 contours compared to the normal F0 condition) were significantly greater in children with CIs than in NH children in all test conditions. Conclusions F0 contours play a major role in sentence recognition in both quiet and noise among pediatric implantees, and the contribution of the F0 contour is even more salient than that in age-matched NH children. These results also suggest that there may be differences between children with CIs and NH children in how F0 contours are processed.

Music perception and training for pediatric cochlear implant users

INTRODUCTION

Cochlear implants (CIs) are biomedical devices that restore sound perception for people with severe-to-profound sensorineural hearing loss. Most postlingually deafened CI users are able to achieve excellent speech recognition in quiet environments. However, current CI sound processors remain limited in their ability to deliver fine spectrotemporal information, making it difficult for CI users to perceive complex sounds. Limited access to complex acoustic cues such as music, environmental sounds, lexical tones, and voice emotion may have significant ramifications on quality of life, social development, and community interactions.

AREAS COVERED

The purpose of this review article is to summarize the literature on CIs and music perception, with an emphasis on music training in pediatric CI recipients. The findings have implications on our understanding of noninvasive, accessible methods for improving auditory processing and may help advance our ability to improve sound quality and performance for implantees.

EXPERT OPINION

Music training, particularly in the pediatric population, may be able to continue to enhance auditory processing even after performance plateaus. The effects of these training programs appear generalizable to non-trained musical tasks, speech prosody and, emotion perception. Future studies should employ rigorous control groups involving a non-musical acoustic intervention, standardized auditory stimuli, and the provision of feedback.

An Eye-Tracking Study on Audiovisual Speech Perception Strategies Adopted by Normal-Hearing and Deaf Adults Under Different Language Familiarities

Purpose The primary purpose of this study was to explore the audiovisual speech perception strategies.80.23.47 adopted by normal-hearing and deaf people in processing familiar and unfamiliar languages. Our primary hypothesis was that they would adopt different perception strategies due to different sensory experiences at an early age, limitations of the physical device, and the developmental gap of language, and others. Method Thirty normal-hearing adults and 33 prelingually deaf adults participated in the study. They were asked to perform judgment and listening tasks while watching videos of a Uygur-Mandarin bilingual speaker in a familiar language (Standard Chinese) or an unfamiliar language (Modern Uygur) while their eye movements were recorded by eye-tracking technology. Results Task had a slight influence on the distribution of selective attention, whereas subject and language had significant influences. To be specific, the normal-hearing and the d10eaf participants mainly gazed at the speaker's eyes and mouth, respectively, in the experiment; moreover, while the normal-hearing participants had to stare longer at the speaker's mouth when they confronted with the unfamiliar language Modern Uygur, the deaf participant did not change their attention allocation pattern when perceiving the two languages. Conclusions Normal-hearing and deaf adults adopt different audiovisual speech perception strategies: Normal-hearing adults mainly look at the eyes, and deaf adults mainly look at the mouth. Additionally, language and task can also modulate the speech perception strategy.

Asynchronous Event-driven Encoder With Simultaneous Temporal Envelope and Phase Extraction for Cochlear Implants

Conventional cochlear implants using periodic sampling are power consuming and incapable of capturing the amplitude and phase of the input acoustic signal simultaneously. This paper presents an asynchronous event-driven encoder chip for cochlear implants capable of extracting the temporal fine structure. The chip architecture is based on asynchronous delta modulation (ADM) where the signal peak/trough crossing events are captured and digitized intrinsically, which has the advantages of significantly reduced power consumption, reduced circuit area, and the elimination of dedicated data compression circuitry. An 8-channel prototype chip was fabricated in 0.18 μm 1P6M CMOS process, occupying an area of 0.125 × 1.7 mm² and has a power consumption of 36.2 μW from a 0.6V supply. A 16-channel stimulation encoding system was built by integrating two test chips, capable of processing the entire audible frequency range from 100 Hz to 10 kHz. Experimental characterization using the human voice is provided to corroborate functionality in the application environment.

Music Training Can Improve Music and Speech Perception in Pediatric Mandarin-Speaking Cochlear Implant Users

Due to limited spectral resolution, cochlear implants (CIs) do not convey pitch information very well. Pitch cues are important for perception of music and tonal language; it is possible that music training may improve performance in both listening tasks. In this study, we investigated music training outcomes in terms of perception of music, lexical tones, and sentences in 22 young (4.8 to 9.3 years old), prelingually deaf Mandarin-speaking CI users. Music perception was measured using a melodic contour identification (MCI) task. Speech perception was measured for lexical tones and sentences presented in quiet. Subjects received 8 weeks of MCI training using pitch ranges not used for testing. Music and speech perception were measured at 2, 4, and 8 weeks after training was begun; follow-up measures were made 4 weeks after training was stopped. Mean baseline performance was 33.2%, 76.9%, and 45.8% correct for MCI, lexical tone recognition, and sentence recognition, respectively. After 8 weeks of MCI training, mean performance significantly improved by 22.9, 14.4, and 14.5 percentage points for MCI, lexical tone recognition, and sentence recognition, respectively (p < .05 in all cases). Four weeks after training was stopped, there was no significant change in posttraining music and speech performance. The results suggest that music training can significantly improve pediatric Mandarin-speaking CI users’ music and speech perception.

Electric and Acoustic Stimulation in Cochlear Implant Recipients with Hearing Preservation

Hearing loss affects 30 million people in the United States, and a subset of these patients have normal low-frequency hearing and ski-sloped high-frequency hearing loss. For these patients, hearing aids alone may not provide adequate benefit. Cochlear implantation alone has been utilized to improve speech perception. The addition of high-frequency electric hearing to low-frequency acoustic hearing in these patients is beneficial. Technical improvements have allowed preservation of low-frequency hearing in cochlear implant recipients, allowing for electric and acoustic stimulation in the same ear with significant improvements in speech perception, sound localization, music appreciation, and quality of life.

A Follow-Up Study on Music and Lexical Tone Perception in Adult Mandarin-Speaking Cochlear Implant Users

OBJECTIVE

The aim was to evaluate the development of music and lexical tone perception in Mandarin-speaking adult cochlear implant (CI) users over a period of 1 year.

STUDY DESIGN

Prospective patient series.

SETTING

Tertiary hospital and research institute.

PATIENTS

Twenty five adult CI users, with ages ranging from 19 to 75 years old, participated in a year-long follow-up evaluation. There were also 40 normal hearing adult subjects who participated as a control group to provide the normal value range.

INTERVENTIONS

Musical sounds in cochlear implants (Mu.S.I.C.) test battery was undertaken to evaluate music perception ability. Mandarin Tone Identification in Noise Test (M-TINT) was used to assess lexical tone recognition. The tests for CI users were completed at 1, 3, 6, and 12 months after the CI switch-on.

MAIN OUTCOMES MEASURES

Quantitative and statistical analysis of their results from music and tone perception tests.

RESULTS

The performance of music perception and tone recognition both demonstrated an overall improvement in outcomes during the entire 1-year follow-up process. The increasing trends were obvious in the early period especially in the first 6 months after switch-on. There was a significant improvement in the melody discrimination (p < 0.01), timbre identification (p < 0.001), tone recognition in quiet (p < 0.0001), and in noise (p < 0.0001).

CONCLUSIONS

Adult Mandarin-speaking CI users show an increasingly improved performance on music and tone perception during the 1-year follow-up. The improvement was the most prominent in the first 6 months of CI use. It is essential to strengthen the rehabilitation training within the first 6 months.

Vocal Emotion Identification by Children Using Cochlear Implants, Relations to Voice Quality, and Musical Interests

PURPOSE

Listening tests for emotion identification were conducted with 8-17-year-old children with hearing impairment (HI; N = 25) using cochlear implants, and their 12-year-old peers with normal hearing (N = 18). The study examined the impact of musical interests and acoustics of the stimuli on correct emotion identification.

METHOD

The children completed a questionnaire with their background information and noting musical interests. They then listened to vocal stimuli produced by actors (N = 5) and consisting of nonsense sentences and prolonged vowels ([a:], [i:], and [u:]; N = 32) expressing excitement, anger, contentment, and fear. The children's task was to identify the emotions they heard in the sample by choosing from the provided options. Acoustics of the samples were studied using Praat software, and statistics were examined using SPSS 24 software.

RESULTS

The children with HI identified the emotions with 57% accuracy and the normal hearing children with 75% accuracy. Female listeners were more accurate than male listeners in both groups. Those who were implanted before age of 3 years identified emotions more accurately than others (p < .05). No connection between the child's audiogram and correct identification was observed. Musical interests and voice quality parameters were found to be related to correct identification.

CONCLUSIONS

Implantation age, musical interests, and voice quality tended to have an impact on correct emotion identification. Thus, in developing the cochlear implants, it may be worth paying attention to the acoustic structures of vocal emotional expressions, especially the formant frequency of F3. Supporting the musical interests of children with HI may help their emotional development and improve their social lives.

What Can Lexical Tone Training Studies in Adults Tell Us about Tone Processing in Children?

A growing number of studies on the acquisition of lexical tone by adult learners have revealed that factors such as language background, musical experience, cognitive abilities, and neuroanatomy all play a role in determining tone learning success. On the basis of these findings, it has been argued that the effectiveness of tone learning in adulthood depends on individual differences in these factors. However, it is not clear whether similar individual differences play an analogous role in tone learning in childhood. Indeed, relatively few studies have made comparisons between how adults and children learn lexical tones. Here, we review recent developments for tone learning in both adults and children. The review covers tone training in a range of contexts, including in naive listeners, in native speakers of other tone languages, in listeners with varying levels of musical experience, and in individuals with speech and hearing disorders. Finally, we discuss the parallels between adult and child tone learning, and provide recommendations concerning how findings in adult tone training can provide insights into tone learning for children by accommodating the needs of individual learners.

Electro-tactile stimulation (ETS) enhances cochlear-implant Mandarin tone recognition

Objective

Electro-acoustic stimulation (EAS) is an effective method to enhance cochlear-implant performance in individuals who have residual low-frequency acoustic hearing. To help the majority of cochlear implant users who do not have any functional residual acoustic hearing, electro-tactile stimulation (ETS) may be used because tactile sensation has a frequency range and perceptual capabilities similar to that produced by acoustic stimulation in the EAS users.

Methods

Following up the first ETS study showing enhanced English sentence recognition in noise,¹ the present study evaluated the effect of ETS on Mandarin tone recognition in noise in two groups of adult Mandarin-speaking individuals. The first group included 11 normal-hearing individuals who listened to a 4-channel, noise-vocoded, cochlear-implant simulation. The second group included 1 unilateral cochlear-implant user and 2 bilateral users with each of their devices being tested independently. Both groups participated in a 4-alternative, forced-choice task, in which they had to identify a tone that was presented in noise at a 0-dB signal-to-noise ratio via electric stimulation (actual or simulated cochlear implants), tactile stimulation or the combined ETS.

Results

While electric or tactile stimulation alone produced similar tone recognition (∼40% correct), the ETS enhanced the cochlear-implant tone recognition by 17–18 percentage points. The size of the present ETS enhancement effect was similar to that of the previously reported EAS effect on Mandarin tone recognition. Psychophysical analysis on tactile sensation showed an important role of frequency discrimination in the ETS enhancement.

Conclusion

Tactile stimulation can potentially enhance Mandarin tone recognition in cochlear-implant users who do not have usable residual acoustic hearing. To optimize this potential, high fundamental frequencies need to be transposed to a 100–200 Hz range.

Cantonese Tone Perception for Children Who Use a Hearing Aid and a Cochlear Implant in Opposite Ears

OBJECTIVES

The ability to recognize tones is vital for speech perception in tonal languages. Cantonese has six tones, which are differentiated almost exclusively by pitch cues (tones 1 to 6). The differences in pitch contours among the tones are subtle, making Cantonese a challenging language for cochlear implant users. The addition of a hearing aid has been shown to improve speech perception in nontonal languages and in Mandarin Chinese. This study (1) investigates the Cantonese tone perception ability of children who use a cochlear implant and a hearing aid in opposite ears; (2) evaluates the effect of varying pitch height and pitch contour cues on Cantonese tone perception for these children; and (3) compares the Cantonese tone perception ability for using a hearing aid and a cochlear implant together versus an implant alone.

DESIGN

Eight native Cantonese speaking children using a cochlear implant and a hearing aid in opposite ears were assessed for tone perception and word identification. The tone perception test involved discriminating and ranking tone pairs from natural and artificially manipulated Cantonese tones with various pitch heights and/or pitch contours. The word identification test involved identifying Cantonese words in a four-alternative forced-choice task. All tests were performed in two device conditions: (1) cochlear implant and hearing aid together and (2) implant alone.

RESULTS

Seven of the 8 subjects performed significantly above chance in both tests using the cochlear implant alone. Results showed that both pitch height and/or pitch direction were important perceptual cues for implant users. Perception for some tones was improved by increasing the pitch height differences between the tones. The ability to discriminate and rank the tone 2/tone 5 contrast and the tone 4/tone 6 contrast was poor, as the tones in these contrasts are similar in pitch contours and onset frequencies. No significant improvement was observed after artificially increasing the pitch offset differences between the tones in the tone 2/tone 5 and the tone 4/tone 6 contrasts. Tone perception results were significantly better with the addition of the hearing aid in the nonimplanted ear compared with using the implant alone; however, word identification results were not significantly different between using the implant alone and using both the hearing aid and the implant together. None of the subjects performed worse in tone perception or in word identification when the hearing aid was added.

CONCLUSIONS

Reduced ability to perceive pitch contour cues, even when artificially exaggerated, may explain some of the difficulties in Cantonese word recognition for implant users. The addition of a contralateral hearing aid could be beneficial for Cantonese tone perception for some individuals with a unilateral implant. The results encouraged Cantonese speakers to trial a hearing aid in the nonimplanted ear when using a cochlear implant.

The Relative Weight of Temporal Envelope Cues in Different Frequency Regions for Mandarin Sentence Recognition

Acoustic temporal envelope (E) cues containing speech information are distributed across the frequency spectrum. To investigate the relative weight of E cues in different frequency regions for Mandarin sentence recognition, E information was extracted from 30 contiguous bands across the range of 80–7,562 Hz using Hilbert decomposition and then allocated to five frequency regions. Recognition scores were obtained with acoustic E cues from 1 or 2 random regions from 40 normal-hearing listeners. While the recognition scores ranged from 8.2% to 16.3% when E information from only one region was available, the scores ranged from 57.9% to 87.7% when E information from two frequency regions was presented, suggesting a synergistic effect among the temporal E cues in different frequency regions. Next, the relative contributions of the E information from the five frequency regions to sentence perception were computed using a least-squares approach. The results demonstrated that, for Mandarin Chinese, a tonal language, the temporal E cues of Frequency Region 1 (80–502 Hz) and Region 3 (1,022–1,913 Hz) contributed more to the intelligence of sentence recognition than other regions, particularly the region of 80–502 Hz, which contained fundamental frequency (F₀) information.

Voice emotion perception and production in cochlear implant users

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Low-frequency fine-structure cues allow for the online use of lexical stress during spoken-word recognition in spectrally degraded speech

English listeners use suprasegmental cues to lexical stress during spoken-word recognition. Prosodic cues are, however, less salient in spectrally degraded speech, as provided by cochlear implants. The present study examined how spectral degradation with and without low-frequency fine-structure information affects normal-hearing listeners' ability to benefit from suprasegmental cues to lexical stress in online spoken-word recognition. To simulate electric hearing, an eight-channel vocoder spectrally degraded the stimuli while preserving temporal envelope information. Additional lowpass-filtered speech was presented to the opposite ear to simulate bimodal hearing. Using a visual world paradigm, listeners' eye fixations to four printed words (target, competitor, two distractors) were tracked, while hearing a word. The target and competitor overlapped segmentally in their first two syllables but mismatched suprasegmentally in their first syllables, as the initial syllable received primary stress in one word and secondary stress in the other (e.g., "'admiral," "'admi'ration"). In the vocoder-only condition, listeners were unable to use lexical stress to recognize targets before segmental information disambiguated them from competitors. With additional lowpass-filtered speech, however, listeners efficiently processed prosodic information to speed up online word recognition. Low-frequency fine-structure cues in simulated bimodal hearing allowed listeners to benefit from suprasegmental cues to lexical stress during word recognition.

Auditory Discrimination of Lexical Stress Patterns in Hearing-Impaired Infants with Cochlear Implants Compared with Normal Hearing: Influence of Acoustic Cues and Listening Experience to the Ambient Language

OBJECTIVES

To assess discrimination of lexical stress pattern in infants with cochlear implant (CI) compared with infants with normal hearing (NH). While criteria for cochlear implantation have expanded to infants as young as 6 months, little is known regarding infants' processing of suprasegmental-prosodic cues which are known to be important for the first stages of language acquisition. Lexical stress is an example of such a cue, which, in hearing infants, has been shown to assist in segmenting words from fluent speech and in distinguishing between words that differ only the stress pattern. To date, however, there are no data on the ability of infants with CIs to perceive lexical stress. Such information will provide insight to the speech characteristics that are available to these infants in their first steps of language acquisition. This is of particular interest given the known limitations that the CI device has in transmitting speech information that is mediated by changes in fundamental frequency.

DESIGN

Two groups of infants participated in this study. The first group included 20 profoundly hearing-impaired infants with CI, 12 to 33 months old, implanted under the age of 2.5 years (median age of implantation = 14.5 months), with 1 to 6 months of CI use (mean = 2.7 months) and no known additional problems. The second group of infants included 48 NH infants, 11 to 14 months old with normal development and no known risk factors for developmental delays. Infants were tested on their ability to discriminate between nonsense words that differed on their stress pattern only (/dóti/ versus /dotí/ and /dotí/ versus /dóti/) using the visual habituation procedure. The measure for discrimination was the change in looking time between the last habituation trial (e.g., /dóti/) and the novel trial (e.g., /dotí/).

RESULTS

(1) Infants with CI showed discrimination between lexical stress pattern with only limited auditory experience with their implant device, (2) discrimination of stress patterns in infants with CI was reduced compared with that of infants with NH, (3) both groups showed directional asymmetry in discrimination, that is, increased discrimination from the uncommon to the common stress pattern in Hebrew (/dóti/ versus /dotí/) compared with the reversed condition.

CONCLUSIONS

The CI device transmitted sufficient acoustic information (amplitude, duration, and fundamental frequency) to allow discrimination between stress patterns in young hearing-impaired infants with CI. The present pattern of results is in support of a discrimination model in which both auditory capabilities and "top-down" interactions are involved. That is, the CI infants detected changes between stressed and unstressed syllables after which they developed a bias for the more common weak-strong stress pattern in Hebrew. The latter suggests that infants with CI were able to extract the statistical distribution of stress patterns by listening to the ambient language even after limited auditory experience with the CI device. To conclude, in relation to processing of lexical stress patterns, infants with CI followed similar developmental milestones as hearing infants thus establishing important prerequisites for early language acquisition.

Mandarin Tone Identification in Cochlear Implant Users Using Exaggerated Pitch Contours

OBJECTIVE

To determine whether exaggerating the variations in fundamental frequency (F0) contours of Mandarin-based pitch fluctuations could improve tone identification by cochlear implant (CI) users.

METHODS

Twelve normal-hearing (NH) listeners and 11 CI users were tested for their ability to recognize F0 contours modeled after Mandarin tones, in 4- or 5-alternatives forced-choice paradigms. Two types of stimuli were used: computer-generated complex tones and voice recordings. Four contours were tested with voice recordings: flat, rise, fall, and dip. A fifth contour, peak, was added for complex tones. The F0 range of each contour was varied in an adaptive manner. A maximum-likelihood technique was used to fit a psychometric function to the performance data and extract threshold at 70% accuracy.

RESULTS

As F0 range increased, performance in tone identification improved but did not reach 100% for some CI users, suggesting that confusions between contours could always be made even with extremely exaggerated contours. Compared with NH participants, CI users required substantially larger F0 ranges to identify tones, on the order of 9.3 versus 0.4 semitones. CI users achieved better performance for complex tones than for voice recordings, whereas the reverse was true for NH participants. Confusion matrices showed that the "flat" tone was often a default option when the tone contour's F0 range presented was too narrow for participants to respond correctly.

CONCLUSION

These results demonstrate markedly impaired ability for CI users to identify tonal contours, but suggest that the use of exaggerated pitch contours may be helpful for tonal language perception.

Mandarin Lexical Tone Acquisition in Cochlear Implant Users With Prelingual Deafness: A Review

PURPOSE

The purpose of this review article is to synthesize evidence from the fields of developmental linguistics and cochlear implant technology relevant to the production and perception of Mandarin lexical tone in cochlear implant users with prelingual deafness. The aim of this review was to identify potential factors that determine outcomes for tonal-language speaking cochlear implant users and possible directions for further research.

METHOD

A computerized database search of MEDLINE, CINAHL, Academic Search Premier, Web of Science, and Google Scholar was undertaken in June and July 2014. Search terms used were lexical tone AND tonal language, speech development AND/OR speech production AND/OR speech perception AND cochlear implants, and pitch perception AND cochlear implants, anywhere in the title or abstract.

CONCLUSION

Despite the demonstrated limitations of pitch perception in cochlear implant users, there is some evidence that typical production and perception of lexical tone is possible by cochlear implant users with prelingual deafness. Further studies are required to determine the factors that contribute to better outcomes to inform rehabilitation processes for cochlear implant users in tonal-language environments.

Pitch and lexical tone perception of bilingual English-Mandarin-speaking cochlear implant recipients, hearing aid users, and normally hearing listeners

OBJECTIVES

The purpose of this current study was to investigate whether pitch, lexical tone, and/or speech-in-noise perception were significantly correlated for Singaporean teenagers or adults who spoke both Mandarin and English.

METHODS

Thirty-three normal hearing or near-normal hearing listeners who did not use a hearing device (NNH group), eight postlingually deafened cochlear implant (CI) recipients (CI group), and three postlingually deafened bilateral hearing aid (HA) users (HA group) were recruited. All participants were bilingual Mandarin-English-speaking Singaporean residents. Participants were assessed on tests of pitch-ranking, lexical tone perception, and speech-in-noise.

RESULTS

The NNH group scored significantly better than the CI group for all tests and subtests. There were no significant differences for the pitch test between the HA group and either the CI or NNH group. However, HA users scored significantly better than the CI group, and more aligned with the NNH group's scores for both the lexical tone and Mandarin speech-in-noise test. There were highly significant moderate positive correlations between all three tests. Discussion Overall, the performance of the CI users in this study indicates that CI recipients still struggle on pitch-related auditory perception tasks. Additionally, although the test scores from the HA users were better than the CI recipients, they were not as good as the NNH listeners. The significant moderate correlations between all three tests indicate that there is at least some degree of overlap in the skills required to accurately perceive these stimuli.

CONCLUSION

The overall results suggest that CI users, and to a lesser extent HA users, still struggle with complex auditory perceptual tasks, particularly when it requires the perception of pitch. However, it may be possible that training one of these skills (e.g. musical pitch) may then generalize to other tasks (e.g. lexical tone and/or speech-in-noise). This is important for counseling, as well as for planning effective rehabilitation programs.

Loudness Contour Can Influence Mandarin Tone Recognition: Vocoder Simulation and Cochlear Implants

Lexical tone recognition with current cochlear implants (CI) remains unsatisfactory due to significantly degraded pitch-related acoustic cues, which dominate the tone recognition by normal-hearing (NH) listeners. Several secondary cues (e.g., amplitude contour, duration, and spectral envelope) that influence tone recognition in NH listeners and CI users have been studied. This work proposes a loudness contour manipulation algorithm, namely Loudness-Tone (L-Tone), to investigate the effects of loudness contour on Mandarin tone recognition and the effectiveness of using loudness cue to enhance tone recognition for CI users. With L-Tone, the intensity of sound samples is multiplied by gain values determined by instantaneous fundamental frequencies (F0s) and pre-defined gain- F0 mapping functions. Perceptual experiments were conducted with a four-channel noise-band vocoder simulation in NH listeners and with CI users. The results suggested that 1) loudness contour is a useful secondary cue for Mandarin tone recognition, especially when pitch cues are significantly degraded; 2) L-Tone can be used to improve Mandarin tone recognition in both simulated and actual CI-hearing without significant negative effect on vowel and consonant recognition. L-Tone is a promising algorithm for incorporation into real-time CI processing and off-line CI rehabilitation training software.

Results using the OPAL strategy in Mandarin speaking cochlear implant recipients

OBJECTIVE

To evaluate the effectiveness of an experimental pitch-coding strategy for improving recognition of Mandarin lexical tone in cochlear implant (CI) recipients.

DESIGN

Adult CI recipients were tested on recognition of Mandarin tones in quiet and speech-shaped noise at a signal-to-noise ratio of +10 dB; Mandarin sentence speech-reception threshold (SRT) in speech-shaped noise; and pitch discrimination of synthetic complex-harmonic tones in quiet. Two versions of the experimental strategy were examined: (OPAL) linear (1:1) mapping of fundamental frequency (F0) to the coded modulation rate; and (OPAL+) transposed mapping of high F0s to a lower coded rate. Outcomes were compared to results using the clinical ACE™ strategy.

STUDY SAMPLE

Five Mandarin speaking users of Nucleus® cochlear implants.

RESULTS

A small but significant benefit in recognition of lexical tones was observed using OPAL compared to ACE in noise, but not in quiet, and not for OPAL+ compared to ACE or OPAL in quiet or noise. Sentence SRTs were significantly better using OPAL+ and comparable using OPAL to those using ACE. No differences in pitch discrimination thresholds were observed across strategies.

CONCLUSIONS

OPAL can provide benefits to Mandarin lexical tone recognition in moderately noisy conditions and preserve perception of Mandarin sentences in challenging noise conditions.

Speech performance in pediatric users of Nurotron ® Venus™ cochlear implants

OBJECTIVES

The primary purpose of this study was to investigate the longitudinal speech performance over 3 years in pediatric users of the Nurotron(®) cochlear implant system. The secondary purpose was to compare the speech performances of younger and older children with cochlear implants (CIs).

METHODS

The Mandarin Early Speech Perception (MESP), Meaningful Use of Speech Scale (MUSS), and Putonghua Chinese Communicative Development Inventory (PCDI) were used to evaluate speech performance of 22 Mandarin-speaking pediatric CI users throughout the first 36 months post-implantation. The subjects were grouped according to the age at implantation, i.e., younger CI group (<3 years) and older CI group (>3 years).

RESULTS

All the subjects demonstrated improvement in speech performance throughout the first 3 years of implant use with mean scores reaching the maximum performance at 36 months post-implantation. The median categories of MESP increased from 0.23 pre-implantation to 5.57 three years post-implantation. Likewise, the median percentage of MUSS was 5.57% to 73.75%; the median performance of PCDI was 55 to 400 for PCDI-comprehension and 32 to 384 for PCDI-production at the same interval. At nearly all test intervals, the older group performed better than the younger group except 24 months post-implantation, at which the MUSS score of the younger CI group was higher than that of the older CI group.

CONCLUSION

The children with Nurotron(®) Venus™ CI system showed considerable gains in speech and language development including tone performance which improved with hearing age. Earlier implantations haven't presented significantly positive performances until 24 months post-implantation in all the tests.

Voice emotion recognition by cochlear-implanted children and their normally-hearing peers

Despite their remarkable success in bringing spoken language to hearing impaired listeners, the signal transmitted through cochlear implants (CIs) remains impoverished in spectro-temporal fine structure. As a consequence, pitch-dominant information such as voice emotion, is diminished. For young children, the ability to correctly identify the mood/intent of the speaker (which may not always be visible in their facial expression) is an important aspect of social and linguistic development. Previous work in the field has shown that children with cochlear implants (cCI) have significant deficits in voice emotion recognition relative to their normally hearing peers (cNH). Here, we report on voice emotion recognition by a cohort of 36 school-aged cCI. Additionally, we provide for the first time, a comparison of their performance to that of cNH and NH adults (aNH) listening to CI simulations of the same stimuli. We also provide comparisons to the performance of adult listeners with CIs (aCI), most of whom learned language primarily through normal acoustic hearing. Results indicate that, despite strong variability, on average, cCI perform similarly to their adult counterparts; that both groups' mean performance is similar to aNHs' performance with 8-channel noise-vocoded speech; that cNH achieve excellent scores in voice emotion recognition with full-spectrum speech, but on average, show significantly poorer scores than aNH with 8-channel noise-vocoded speech. A strong developmental effect was observed in the cNH with noise-vocoded speech in this task. These results point to the considerable benefit obtained by cochlear-implanted children from their devices, but also underscore the need for further research and development in this important and neglected area. This article is part of a Special Issue entitled .

The effect of enhancing temporal periodicity cues on Cantonese tone recognition by cochlear implantees

OBJECTIVES

This study investigates the efficacy of a cochlear implant (CI) processing method that enhances temporal periodicity cues of speech.

DESIGN

Subjects participated in word and tone identification tasks. Two processing conditions - the conventional advanced combination encoder (ACE) and tone-enhanced ACE were tested. Test materials were Cantonese disyllabic words recorded from one male and one female speaker. Speech-shaped noise was added to clean speech. The fundamental frequency information for periodicity enhancement was extracted from the clean speech. Electrical stimuli generated from the noisy speech with and without periodicity enhancement were presented via direct stimulation using a Laura 34 research processor. Subjects were asked to identify the presented word.

STUDY SAMPLE

Seven post-lingually deafened native Cantonese-speaking CI users.

RESULTS

Percent correct word, segmental structure, and tone identification scores were calculated. While word and segmental structure identification accuracy remained similar between the two processing conditions, tone identification in noise was better with tone-enhanced ACE than with conventional ACE. Significant improvement on tone perception was found only for the female voice.

CONCLUSIONS

Temporal periodicity cues are important to tone perception in noise. Pitch and tone perception by CI users could be improved when listeners received enhanced temporal periodicity cues.

Effects of lexical tone contour on Mandarin sentence intelligibility

PURPOSE

This study examined the effects of lexical tone contour on the intelligibility of Mandarin sentences in quiet and in noise.

METHOD

A text-to-speech synthesis engine was used to synthesize Mandarin sentences with each word carrying the original lexical tone, flat tone, or a tone randomly selected from the 4 Mandarin lexical tones. The synthesized speech signals were presented to 11 normal-hearing listeners for recognition in quiet and in speech-shaped noise at 0 dB signal-to-noise ratio.

RESULTS

Normal-hearing listeners nearly perfectly recognized the Mandarin sentences produced with modified tone contours in quiet; however, performance declined substantially in noise.

CONCLUSIONS

Consistent with previous findings to some extent, the present findings suggest that lexical tones are relatively redundant cues for Mandarin sentence intelligibility in quiet and that other cues could compensate for the distorted lexical tone contour. However, in noise, the results provide direct evidence that lexical tone contour is important for the recognition of Mandarin sentences.

Perception of vowels and prosody by cochlear implant recipients in noise

The aim of the present study was to compare the ability of cochlear implant (CI) recipients to recognise speech prosody in the presence of speech-weighted noise to their ability to recognise vowels in the same test paradigm and listening condition. All test materials were recorded from four different speakers (two male, two female). Two prosody recognition tasks were developed, both using single words as stimuli. The first task involved a question/statement distinction, while the second task required listeners to make a judgement about the speaker's attitude. Vowel recognition tests were conducted using vowel pairs selected on the basis of specific acoustic cues (frequencies of the first two formants and duration). Ten CI users and ten normal-hearing controls were tested in both quiet and an adaptive noise condition, using a two-alternative forced-choice test paradigm for all the tests. Results indicated that vowel recognition was significantly better than prosody recognition in both listener groups in both quiet and noise, and that question/statement discrimination was the most difficult task for CI listeners in noise. Data from acoustic analyses were used to interpret differences in performance on different tasks and with different speakers.

LEARNING OUTCOMES

As a result of this activity, readers will be able to (1) describe suitable methods for comparing vowel and prosody perception in noise, (2) compare performance on vowel and prosody perception tasks in quiet in normal-hearing listeners and cochlear implant recipients, (3) compare performance on vowel and prosody perception tasks in noise in normal-hearing listeners and cochlear implant recipients and (4) relate performance on prosody tasks in quiet to performance on these tasks in noise.

Speech perception and communication ability over the telephone by Mandarin-speaking children with cochlear implants

OBJECTIVE

(1) To understand speech perception and communication ability through real telephone calls by Mandarin-speaking children with cochlear implants and compare them to live-voice perception, (2) to report the general condition of telephone use of this population, and (3) to investigate the factors that correlate with telephone speech perception performance.

METHODS

Fifty-six children with over 4 years of implant use (aged 6.8-13.6 years, mean duration 8.0 years) took three speech perception tests administered using telephone and live voice to examine sentence, monosyllabic-word and Mandarin tone perception. The children also filled out a questionnaire survey investigating everyday telephone use. Wilcoxon signed-rank test was used to compare the scores between live-voice and telephone tests, and Pearson's test to examine the correlation between them.

RESULTS

The mean scores were 86.4%, 69.8% and 70.5% respectively for sentence, word and tone recognition over the telephone. The corresponding live-voice mean scores were 94.3%, 84.0% and 70.8%. Wilcoxon signed-rank test showed the sentence and word scores were significantly different between telephone and live voice test, while the tone recognition scores were not, indicating tone perception was less worsened by telephone transmission than words and sentences. Spearman's test showed that chronological age and duration of implant use were weakly correlated with the perception test scores. The questionnaire survey showed 78% of the children could initiate phone calls and 59% could use the telephone 2 years after implantation.

CONCLUSION

Implanted children are potentially capable of using the telephone 2 years after implantation, and communication ability over the telephone becomes satisfactory 4 years after implantation.

Cochlear implants with fine structure processing improve speech and tone perception in Mandarin-speaking adults

CONCLUSION

This study indicates that Mandarin-speaking subjects acquire significant benefit from a cochlear implant (CI) and audio processor with the fine structure processing (FSP) coding strategy. It can take adult Mandarin CI users time to become accustomed to their CI and appreciate the benefits.

OBJECTIVES

This study aimed to determine speech perception outcomes in Mandarin-speaking adults who received the FSP coding strategy.

METHODS

This was a prospective study. Ten adults who were implanted unilaterally with a PULSARCI(100), and received an OPUS 1 or OPUS 2 speech processor, were included in this study. The mean age at implantation was 31.1 years. We determined the benefit to postlingually deafened Mandarin-speaking adults with a severe to profound hearing loss using the tests: monosyllables in quiet, sentences in quiet and the Mandarin Hearing in Noise Test (MHINT). Tone perception was evaluated using a tone perception test. A visual analog scale (VAS) was used to score sound quality and ease of listening.

RESULTS

The results showed a significant improvement between first fitting and 6 months in all speech tests and in tone perception. The VAS scores showed a significant improvement over 6 months in sound quality and an immediate benefit in ease of listening.

Mandarin lexical tone recognition in sensorineural hearing-impaired listeners and cochlear implant users

CONCLUSIONS

As the hearing loss becomes more severe, the tone recognition performance of hearing-impaired listeners gradually but slowly reduces. The tone recognition performance of cochlear implant listeners is below or close to the performance of severely hearing-impaired listeners.

OBJECTIVES

The present study aimed to investigate the Mandarin lexical tone recognition performance of sensorineural hearing-impaired listeners and post-lingually deafened cochlear implant users.

METHODS

Tone recognition performance was measured for 30 normal-hearing subjects, 41sensorineural hearing-impaired listeners, and 12 cochlear implant users using 128 monosyllables recorded by a male and a female adult native Mandarin speaker.

RESULTS

The results indicated that the accuracy of tone recognition was 99.3%, 96.4%, 93.7%, 83.9%, and 81.0% for the normal-hearing, moderate, moderate to severe, severely hearing-impaired, and cochlear implant subjects, respectively. For the hearing-impaired subjects, a significantly negative correlation was observed between tone recognition performance and the audiometric hearing thresholds. For cochlear implant subjects, Tone 3 was the easiest one to perceive and Tone 2 was the hardest one to perceive. They tended to misperceive Tone 1 as Tone 2, and misperceive Tone 2 as Tones 1 and 3.

The effects of binaural spectral resolution mismatch on Mandarin speech perception in simulated electric hearing

This study assessed the effects of binaural spectral resolution mismatch on the intelligibility of Mandarin speech in noise using bilateral cochlear implant simulations. Noise-vocoded Mandarin speech, corrupted by speech-shaped noise at 0 and 5 dB signal-to-noise ratios, were presented unilaterally or bilaterally to normal-hearing listeners with mismatched spectral resolution between ears. Significant binaural benefits for Mandarin speech recognition were observed only with matched spectral resolution between ears. In addition, the performance of tone identification was more robust to noise than that of sentence recognition, suggesting factors other than tone identification might account more for the degraded sentence recognition in noise.

Acoustic cue integration in speech intonation recognition with cochlear implants

The present article reports on the perceptual weighting of prosodic cues in question-statement identification by adult cochlear implant (CI) listeners. Acoustic analyses of normal-hearing (NH) listeners' production of sentences spoken as questions or statements confirmed that in English the last bisyllabic word in a sentence carries the dominant cues (F0, duration, and intensity patterns) for the contrast. Furthermore, these analyses showed that the F0 contour is the primary cue for the question-statement contrast, with intensity and duration changes conveying important but less reliable information. On the basis of these acoustic findings, the authors examined adult CI listeners' performance in two question-statement identification tasks. In Task 1, 13 CI listeners' question-statement identification accuracy was measured using naturally uttered sentences matched for their syntactic structures. In Task 2, the same listeners' perceptual cue weighting in question-statement identification was assessed using resynthesized single-word stimuli, within which fundamental frequency (F0), intensity, and duration properties were systematically manipulated. Both tasks were also conducted with four NH listeners with full-spectrum and noise-band-vocoded stimuli. Perceptual cue weighting was assessed by comparing the estimated coefficients in logistic models fitted to the data. Of the 13 CI listeners, 7 achieved high performance levels in Task 1. The results of Task 2 indicated that multiple sources of acoustic cues for question-statement identification were utilized to different extents depending on the listening conditions (e.g., full spectrum vs. spectrally degraded) or the listeners' hearing and amplification status (e.g., CI vs. NH).

Comparison of Mandarin tone and speech perception between advanced combination encoder and continuous interleaved sampling speech-processing strategies in children

OBJECTIVE

This study was performed to compare cochlear implant (CI) users' performance in Mandarin speech and tone perception between 2 types of speech-processing strategies-advanced combination encoder (ACE) and continuous interleaved sampling (CIS)-under quiet and noisy conditions.

METHODS

This study involved 10 congenitally deaf children (age range, 5.7-15.3 years; mean, 9.2 years) who received the Nucleus 24-channel CI system cochlear device (CI24R; Cochlear Ltd, Lane Cove NSW, Australia). The subjects used ACE since switching on their CI devices. Speech and tone perception tests were administered under quiet and noisy (+5 dB signal-to-noise ratio) conditions with ACE and CIS strategies 20 minutes and 2 weeks apart.

RESULTS

Regardless of the strategy used, subjects showed significantly higher scores in speech perception than in tone recognition. Under noisy conditions, subjects had significantly higher tone identification scores with the CIS than the ACE strategy (P = .038). There was no significant difference in speech identification score between the strategies. Subjects showed significant higher tone identification and speech perception scores under quiet than noisy (+5 dB signal-to-noise ratio) conditions. Subjectively, 6 subjects preferred the ACE strategy, and the remaining 4 preferred the CIS strategy. The strategy preference of the subjects was related to speech perception performance rather than tone identification. A significant correlation was observed between tone identification and speech recognition, regardless of whether speech was evaluated by consonants (r = 0.669, P < .001), vowels (r = 0.426, P = .001), or sentences (r = 0.294, P = .023).

CONCLUSION

There are only 4 patterns of tone in Mandarin, which is far fewer than the number of speech sounds. However, tone identification is poorer than speech perception. The CIS speech-processing strategy may improve tone identification under noisy conditions. Before improved speech strategies to code acoustic characteristics of tone can be developed, it would be worthwhile to try both CIS and ACE for CI users and to select the most suitable speech-processing strategy according to the subjective preference and objective performance.