Combining acoustic and electric stimulation in the service of speech recognition

Timbre and speech perception in bimodal and bilateral cochlear-implant listeners

OBJECTIVES

This study was designed to evaluate the contribution of temporal and spectral cues for timbre perception in listeners with a cochlear implant (CI) in one ear and low-frequency residual hearing in the contralateral ear (bimodal hearing), and listeners with two CIs. Specifically, it examined the relationship between timbre and speech perception in these two groups of listeners. It was hypothesized that, similar to speech recognition, temporal-envelope cues are dominant cues for timbre perception, and the reliance of spectral cues was reduced in both bimodal and bilateral CI users compared with that in normal-hearing listeners. It was further hypothesized that the patterns of results with regard to combined benefit would be similar between timbre and speech perception.

DESIGN

Seven bimodal and five bilateral CI users participated. Sixteen stimuli that synthesized western musical instruments were used for the timbre-perception task. Sixteen consonants in the /aCa/ context and nine monophthongs in the /hVd/ context were used for the phoneme-recognition task. Each subject was tested on three listening conditions-individual device alone (single CI, or hearing aid [HA] alone) and combined use of devices (CI + HA, or 2CIs). For the timbre-perception task, each listener made judgments of dissimilarity between stimulus pairs. Multidimensional scaling analysis was performed to derive the coordinates of the dimensions that best fit the data. Correlational analyses were performed to relate the coordinates of each dimension and the temporal-envelope (impulsiveness) and spectral-envelope (spectral-centroid) features of the stimuli. For phoneme-recognition task, each listener identified the phoneme he or she heard by choosing an answer displayed on the computer screen. Overall percent correct phoneme-identification scores and percent information transmission for consonant and vowel features were calculated.

RESULTS

There were strong correlations between impulsiveness and the first dimension (Dim 1) of the timbre space, but correlations between spectral centroid and the second dimension (Dim 2) were weak for all listening conditions for both groups of listeners. As a group, the combined use of devices did not significantly improve listeners' ability to perceive differences in musical timbre compared with the better single-device condition. Some of the bimodal and bilateral CI users showed a considerably strengthened correlation between spectral centroid and Dim 2 in the combined condition compared with a single CI or an HA. There was a lack of relationship between percent correct phoneme recognition and timbre perception for all listening conditions. However, there was a consistent pattern regarding the combined benefit between timbre perception and vowel recognition. In general, listeners who demonstrated combined benefit for vowel recognition also showed a considerable increase in correlation between spectral centroid and Dim 2 with the combined use of devices compared with the single-device conditions. Improved correlation was not evident for those who did not demonstrate combined benefit for vowel recognition.

CONCLUSIONS

Similar to speech recognition, temporal envelope was a dominant cue for timbre perception in bimodal and bilateral CI users. In addition, there was a close relationship between timbre perception and vowel recognition with regard to combined benefit. The present findings suggest that speech recognition and timbre perception could be enhanced when listeners received different spectral cues from individual devices.

Auditory training in patients with unilateral cochlear implant and contralateral acoustic stimulation

OBJECTIVES

It was hypothesized that auditory training would allow bimodal patients to combine in a better manner the low-frequency acoustic information provided by a hearing aid with the electric information provided by a cochlear implant, thus maximizing the benefit of combining acoustic (A) and electric (E) stimulation (EAS).

DESIGN

Performance in quiet or in the presence of a multitalker babble at +5 dB signal to noise ratio was evaluated in seven bimodal patients before and after auditory training. The performance measures comprised identification of vowels and consonants, consonant-nucleus-consonant words, sentences, voice gender, and emotion. Baseline performance was evaluated in the A-alone, E-alone, and combined EAS conditions once per week for 3 weeks. A phonetic-contrast training protocol was used to facilitate speech perceptual learning. Patients trained at home 1 hour a day, 5 days a week, for 4 weeks with both their cochlear implant and hearing aid devices on. Performance was remeasured after the 4 weeks of training and 1 month after training stopped.

RESULTS

After training, there was significant improvement in vowel, consonant, and consonant-nucleus-consonant word identification in the E and EAS conditions. The magnitude of improvement in the E condition was equivalent to that in the EAS condition. The improved performance was largely retained 1 month after training stopped.

CONCLUSION

Auditory training, in the form administered in this study, can improve bimodal patients' overall speech understanding by improving E-alone performance.

Effects of lower frequency-to-electrode allocations on speech and pitch perception with the hybrid short-electrode cochlear implant

Because some users of a Hybrid short-electrode cochlear implant (CI) lose their low-frequency residual hearing after receiving the CI, we tested whether increasing the CI speech processor frequency allocation range to include lower frequencies improves speech perception in these individuals. A secondary goal was to see if pitch perception changed after experience with the new CI frequency allocation. Three subjects who had lost all residual hearing in the implanted ear were recruited to use an experimental CI frequency allocation with a lower frequency cutoff than their current clinical frequency allocation. Speech and pitch perception results were collected at multiple time points throughout the study. In general, subjects showed little or no improvement for speech recognition with the experimental allocation when the CI was worn with a hearing aid in the contralateral ear. However, all 3 subjects showed changes in pitch perception that followed the changes in frequency allocations over time, consistent with previous studies showing that pitch perception changes upon provision of a CI.

Current research with cochlear implants at Arizona State University

In this article we review, and discuss the clinical implications of, five projects currently underway in the Cochlear Implant Laboratory at Arizona State University. The projects are (1) norming the AzBio sentence test, (2) comparing the performance of bilateral and bimodal cochlear implant (CI) patients in realistic listening environments, (3) accounting for the benefit provided to bimodal patients by low-frequency acoustic stimulation, (4) assessing localization by bilateral hearing aid patients and the implications of that work for hearing preservation patients, and (5) studying heart rate variability as a possible measure for quantifying the stress of listening via an implant. The long-term goals of the laboratory are to improve the performance of patients fit with cochlear implants and to understand the mechanisms, physiological or electronic, that underlie changes in performance. We began our work with cochlear implant patients in the mid-1980s and received our first grant from the National Institutes of Health (NIH) for work with implanted patients in 1989. Since that date our work with cochlear implant patients has been funded continuously by the NIH. In this report we describe some of the research currently being conducted in our laboratory.

THE PSYCHOPHYSICS OF LOW-FREQUENCY ACOUSTIC HEARING IN ELECTRIC AND ACOUSTIC STIMULATION (EAS) AND BIMODAL PATIENTS

This paper provides a review of the current literature on psychophysical properties of low-frequency hearing, both before and after implantation, with a focus on frequency selectivity, nonlinear cochlear processing, and speech perception in temporally modulated maskers for bimodal listeners as well as patients with hearing preservation in the implanted ear and receiving combined electric and acoustic stimulation (EAS). In this paper we review our work, the work of others, and report results not previously published for speech perception in steady-state and temporally fluctuating maskers; the degree of masking release and frequency resolution for 11 bimodal, 6 hearing preservation patients; and 5 control subjects with normal hearing. The results demonstrate that a small masking release is possible with acoustic hearing in just one ear, with the degree of masking release being correlated with the low-frequency pure tone average in the non-implanted ear; furthermore, frequency selectivity as defined by the width of the auditory filter was not correlated with the degree of masking release. Descriptions of the clinical utility of hearing preservation in the implanted ear for improving speech perception in complex listening environments, as well as directions for the future, are discussed.

Advances in auditory prostheses

PURPOSE OF REVIEW

Auditory prostheses use electric currents on multiple electrodes to stimulate auditory neurons and recreate auditory sensations in deaf people. Cochlear implants have restored hearing in more than 200  000 deaf adults and children to a level that allows most to understand speech. Here we review the reasons underlying these results and describe new directions in restoring hearing to additional patient populations and the design of new devices.

RECENT FINDINGS

From their early development about 50 years ago, cochlear implants have been well received and beneficial to people who had lost their hearing. Although those first implants did not allow high levels of speech understanding, they provided auditory information that worked synergistically with lip reading to improve communication. Present day cochlear implants provide excellent speech understanding in children and in postlingually deafened adults. Research is focused on improved signal processing and new electrode designs. Electric stimulation of the auditory brainstem can also produce excellent hearing in some children and adults.

SUMMARY

Auditory prostheses, both at the level of the sensory nerve and at the brainstem, can restore patterns of neural activation that are sufficient for high levels of speech understanding. These prostheses are not only clinically successful but also important tools for understanding sensory processing in the brain.

Bilateral cochlear implantation

Cochlear implantation (CI) is the standard of care for the treatment of children and adults with bilateral severe-to-profound sensorineural hearing loss. Because the ultimate and continuous goal of CI teams is to improve patient performance, a potential method is bilateral CI. The potential benefits of bilateral CI include binaural summation, squelch, equivalent head shadow for each ear, improved hearing in noise, sound localization ability, and spatial release from masking. The potential disadvantages include additional or prolonged surgical procedure, unproven cost/benefit profile, and the elimination of the ability to use future technologies and/or medical therapies in the implanted ear.

Electro acoustic stimulation of the auditory system: UNICAMP's surgical approach

A new era has arrived in auditory rehabilitation with the introduction of new technologies such as electroacoustic stimulation (EAS). EAS is indicated for patients with residual hearing at low frequencies and severe or profound hearing loss at high frequencies. These patients have no indication for conventional cochlear implant and have difficulties in adapting to individual sound amplification devices. Preservation of hearing is vital in this process; the surgical technique must be based on this concept.

Objectives

To present the cochlear implant surgical technique with MED-EL Mand FlexEAS to preserve hearing in patients with hearing loss at high frequencies and to maintain low frequency hearing. We are the first institution to carry out this treatment in Brazil.

Methods

A case report of the surgical technique carried out in four patients; the procedure was carried out by the cochlear implant group of a specialized clinical hospital.

Results

The procedures were successful and uneventful.

Conclusion

We described the technique used at our institution for implants using EAS; the surgical technique is complex and includes steps for preservation of hearing.

BILATERAL AND BIMODAL BENEFITS AS A FUNCTION OF AGE FOR ADULTS FITTED WITH A COCHLEAR IMPLANT

BACKGROUND

Both bilateral cochlear implants (CIs) and bimodal (electric plus contralateral acoustic) stimulation can provide better speech intelligibility than a single CI. In both cases patients need to combine information from two ears into a single percept. In this paper we ask whether the physiological and psychological processes associated with aging alter the ability of bilateral and bimodal CI patients to combine information across two ears in the service of speech understanding.

MATERIALS

The subjects were 61 adult, bilateral CI patients and 94 adult, bimodal patients. The test battery was composed of monosyllabic words presented in quiet and the AzBio sentences presented in quiet, at +10 and at +5 dB signal-to-noise ratio (SNR).

METHODS

The subjects were tested in standard audiometric sound booths. Speech and noise were always presented from a single speaker directly in front of the listener.

RESULTS

Age and bilateral or bimodal benefit were not significantly correlated for any test measure.

CONCLUSIONS

Other factors being equal, both bilateral CIs and bimodal CIs can be recommended for elderly patients.

Potential benefits from deeply inserted cochlear implant electrodes

OBJECTIVES

This review examines evidence for potential benefits of using cochlear implant electrodes that extend into the apical regions of the cochlea. Most cochlear implant systems use electrode arrays that extend 1 to 1.5 turns from the basal cochleostomy, but one manufacturer (MED-EL GmbH) uses an electrode array that is considerably longer. The fundamental rationale for using electrodes extending toward the apex of the cochlea is to provide additional low-pitched auditory percepts and thereby increase the spectral information available to the user. Several experimental long arrays have also been produced by other manufacturers to assess potential benefits of this approach.

DESIGN

In addition to assessing the effects of deeply inserted electrodes on performance, this review examines several underlying and associated issues, including cochlear anatomy, electrode design, surgical considerations (including insertion trauma), and pitch scaling trials. Where possible, the aim is to draw conclusions regarding the potential from apical electrodes in general, rather than relating to the performance of specific and current devices.

RESULTS

Imaging studies indicate that currently available electrode arrays rarely extend more than two turns into the cochlea, the mean insertion angle for full insertions of the MED-EL electrodes being about 630°. This is considerably shorter than the total length of the cochlea and more closely approximates the length of the spiral ganglion. Anatomical considerations, and some modelling studies, suggest that fabrication of even longer electrodes is unlikely to provide additional spectral information. The issue of potential benefit from the most apical electrodes, therefore, is whether they are able to selectively stimulate discrete and tonotopically ordered neural populations near the apex of the spiral ganglion, where the ganglion cells are closely grouped. Pitch scaling studies, using the MED-EL and experimental long arrays, suggest that this is achieved in many cases, but that a significant number of individuals show evidence of pitch confusions or reversals among the most apical electrodes, presumably reducing potential performance benefit and presenting challenges for processor programming.

CONCLUSIONS

Benefits in terms of speech recognition and other performance measures are less clear. Several studies have indicated that deactivation of apical electrodes results in poorer speech recognition performance, but these have been mostly acute studies where the subjects have been accustomed to the full complement of electrodes, thus making interpretation difficult. Some chronic studies have suggested that apical electrodes do provide additional performance benefit, but others have shown performance improvement after deactivating some of the apical electrodes. Whether or not deeply inserted electrodes can offer performance benefits, there is evidence that currently available designs tend to produce more intracochlear trauma than shorter arrays, in terms of loss of residual acoustic hearing and reduction of the neural substrate. This may have important long-term consequences for the user. Furthermore, as it is possible that subjects with better low-frequency residual hearing are more likely to benefit from the inclusion of apical electrodes, there may be a potential clinical dilemma as the same subjects are those most likely to benefit from bimodal electroacoustic stimulation, requiring a relatively shallow insertion.