Psychophysics and Electrical Stimulation. In: Zeng F, Popper AN, Fay RR, editors. Cochlear Implants: Auditory Prostheses and Electric Hearing. New York: Springer; 2004. pp. 286-333. [DOI: 10.1007/978-0-387-22585-2_7]

Perception of Musical Tension in Cochlear Implant Listeners

Despite the difficulties experienced by cochlear implant (CI) users in perceiving pitch and harmony, it is not uncommon to see CI users listening to music, or even playing an instrument. Listening to music is a complex process that relies not only on low-level percepts, such as pitch or timbre, but also on emotional reactions or the ability to perceive musical sequences as patterns of tension and release. CI users engaged in musical activities might experience some of these higher-level musical features. The goal of this study is to evaluate CI users' ability to perceive musical tension. Nine CI listeners (CIL) and nine normal-hearing listeners (NHL) were asked to rate musical tension on a continuous visual analog slider during music listening. The subjects listened to a 4 min recording of Mozart's Piano Sonata No. 4 (K282) performed by an experienced pianist. In addition to the original piece, four modified versions were also tested to identify which features might influence the responses to the music in the two groups. In each version, one musical feature of the piece was altered: tone pitch, intensity, rhythm, or tempo. Surprisingly, CIL and NHL rated overall musical tension in a very similar way in the original piece. However, the results from the different modifications revealed that while NHL ratings were strongly affected by music with random pitch tones (but preserved intensity and timing information), CIL ratings were not. Rating judgments of both groups were similarly affected by modifications of rhythm and tempo. Our study indicates that CI users can understand higher-level musical aspects as indexed by musical tension ratings. The results suggest that although most CI users have difficulties perceiving pitch, additional music cues, such as tempo and dynamics might contribute positively to their experience of music.

Binaural timing information in electric hearing at low rates: Effects of inaccurate encoding and loudness

Stimulation strategies for cochlear implants potentially impose timing limitations that may hinder the correct encoding and representation of interaural time differences (ITDs) in realistic bilateral signals. This study aimed to specify the tolerable room for inaccurate encoding of ITDs at low rates by investigating the perceptual degradation due to the removal of individual pulses at various levels of loudness. Unmodulated, 100-pulses-per-second pulse trains were presented at a single, interaurally pitch-matched electrode pair. In experiment I, ITD thresholds were measured applying different degrees of bilateral, interaurally-uncorrelated pulse removal. The ITD sensitivity deteriorated with increasing degree of pulse removal, with significant deterioration for degrees of 16% or greater. In experiment II, the interaction between loudness and pulse removal was investigated. Louder stimuli yielded better ITD sensitivity, however, no further improvement was found for stimuli louder than "medium." When removing 8% of the pulses, the ITD sensitivity deteriorated significantly across the entire loudness range tested. A loudness-induced compensation for the deterioration of ITD sensitivity due to pulse removal seems to be feasible for soft stimuli but not for medium or loud stimuli. Overall, our findings suggest that the degree of pulse removal employed in low-rate channels within coding strategies should not exceed 8%.

Channel Interaction and Current Level Affect Across-Electrode Integration of Interaural Time Differences in Bilateral Cochlear-Implant Listeners

Sensitivity to interaural time differences (ITDs) is important for sound localization. Normal-hearing listeners benefit from across-frequency processing, as seen with improved ITD thresholds when consistent ITD cues are presented over a range of frequency channels compared with when ITD information is only presented in a single frequency channel. This study aimed to clarify whether cochlear-implant (CI) listeners can make use of similar processing when being stimulated with multiple interaural electrode pairs transmitting consistent ITD information. ITD thresholds for unmodulated, 100-pulse-per-second pulse trains were measured in seven bilateral CI listeners using research interfaces. Consistent ITDs were presented at either one or two electrode pairs at different current levels, allowing for comparisons at either constant level per component electrode or equal overall loudness. Different tonotopic distances between the pairs were tested in order to clarify the potential influence of channel interaction. Comparison of ITD thresholds between double pairs and the respective single pairs revealed systematic effects of tonotopic separation and current level. At constant levels, performance with double-pair stimulation improved compared with single-pair stimulation but only for large tonotopic separation. Comparisons at equal overall loudness revealed no benefit from presenting ITD information at two electrode pairs for any tonotopic spacing. Irrespective of electrode-pair configuration, ITD sensitivity improved with increasing current level. Hence, the improved ITD sensitivity for double pairs found for a large tonotopic separation and constant current levels seems to be due to increased loudness. The overall data suggest that CI listeners can benefit from combining consistent ITD information across multiple electrodes, provided sufficient stimulus levels and that stimulating electrode pairs are widely spaced.

Localization in reverberation with cochlear implants: predicting performance from basic psychophysical measures

Users of bilateral cochlear implants (CIs) experience difficulties localizing sounds in reverberant rooms, even in rooms where normal-hearing listeners would hardly notice the reverberation. We measured the localization ability of seven bilateral CI users listening with their own devices in anechoic space and in a simulated reverberant room. To determine factors affecting performance in reverberant space we measured the sensitivity to interaural time differences (ITDs), interaural level differences (ILDs), and forward masking in the same participants using direct computer control of the electric stimulation in their CIs. Localization performance, quantified by the coefficient of determination r(2) and the root mean squared error, was significantly worse in the reverberant room than in anechoic conditions. Localization performance in the anechoic room, expressed as r(2), was best predicted by subject's sensitivity to ILDs. However, the decrease in localization performance caused by reverberation was better predicted by the sensitivity to envelope ITDs measured on single electrode pairs, with a correlation coefficient of 0.92. The CI users who were highly sensitive to envelope ITDs also better maintained their localization ability in reverberant space. Results in the forward masking task added only marginally to the predictions of localization performance in both environments. The results indicate that envelope ITDs provided by CI processors support localization in reverberant space. Thus, methods that improve perceptual access to envelope ITDs could help improve localization with bilateral CIs in everyday listening situations.

Sensitivity to interaural level difference and loudness growth with bilateral bimodal stimulation

The interaural level difference (ILD) is an important cue for the localization of sound sources. The sensitivity to ILD was measured in 10 users of a cochlear implant (CI) in one ear and a hearing aid (HA) in the other severely impaired ear. For simultaneous presentation of a pulse train on the CI side and a sinusoid on the HA side the just noticeable difference (JND) in ILD and loudness growth functions were measured. The mean JND for pitch-matched electric and acoustic stimulation was 1.7 dB. A linear fit of the loudness growth functions on a decibel-versus-microampere scale shows that the slope depends on the subject's dynamic ranges.

The growth of loudness functions measured in cochlear implant listeners using absolute magnitude estimation and compared using Akaike's information criterion

The input/output function for acoustic hearing can be characterized by the growth of loudness with sound pressure level and generally follows a compressive power law. In contrast, in electric hearing, loudness reportedly is an expansive function of applied electrical current but the specific shape of the function has not been fully determined. Loudness growth models have implications for the implementation of cochlear implant speech processors. Having an appropriate loudness growth model is important to cochlear implant users because they have a small dynamic range of hearing compared to normal hearing listeners. To compensate for this, appropriate models of loudness are necessary for the design of cochlear implant speech processors. It is also necessary to understand how loudness is encoded and may affect the relative performance in speech recognition. Currently, there is no consensus on the actual shape of the loudness growth function, with power or exponential functions being suggested. In this study psychophysical loudness growth measures were obtained in twelve adult cochlear implant listeners, using the method of absolute magnitude estimation and production. Best-fit loudness growth functions as determined by Akaike's Information Criterion (AIC) method for finding the best-fit loudness model seem to show a difference in the loudness growth functions across subjects and across electrode pairs within individual subjects. The range of functions observed is greater than previously reported and goes from linear to expansive, suggesting that individual variations in dynamic range should be incorporated in the design of cochlear implant sound processors.

Current-level discrimination in the context of interleaved, multichannel stimulation in cochlear implants: effects of number of stimulated electrodes, pulse rate, and electrode separation

The ability of cochlear implantees to detect an increment in current level at one of many stimulated electrodes was investigated. Such changes in the electric profile provide information for cochlear implantees to discriminate numerous sounds, especially vowels. In Experiment 1, sensitivity to increases in current level at one stimulation site in the electric profile decreased as the number of stimulated electrodes increased. This outcome was most likely a result of decreased stimulus levels at individual electrodes that were required to retain a comfortable loudness when the number of active electrodes was increased. Experiment 2 investigated the effects of pulse rate and separation between stimulation sites when the levels in percent of dynamic range and number of stimulated electrodes were held constant. The effect of pulse rate and electrode separation varied among listeners. The sensitivity of 6 of 9 listeners was best at the pulse rate that they used clinically. This might have been the result of adaptation to the clinical pulse rate, or listeners might have chosen their inherently best pulse rate during the clinical fitting.

Music perception with cochlear implants: a review

The acceptance of cochlear implantation as an effective and safe treatment for deafness has increased steadily over the past quarter century. The earliest devices were the first implanted prostheses found to be successful in compensating partially for lost sensory function by direct electrical stimulation of nerves. Initially, the main intention was to provide limited auditory sensations to people with profound or total sensorineural hearing impairment in both ears. Although the first cochlear implants aimed to provide patients with little more than awareness of environmental sounds and some cues to assist visual speech-reading, the technology has advanced rapidly. Currently, most people with modern cochlear implant systems can understand speech using the device alone, at least in favorable listening conditions. In recent years, an increasing research effort has been directed towards implant users' perception of nonspeech sounds, especially music. This paper reviews that research, discusses the published experimental results in terms of both psychophysical observations and device function, and concludes with some practical suggestions about how perception of music might be enhanced for implant recipients in the future. The most significant findings of past research are: (1) On average, implant users perceive rhythm about as well as listeners with normal hearing; (2) Even with technically sophisticated multiple-channel sound processors, recognition of melodies, especially without rhythmic or verbal cues, is poor, with performance at little better than chance levels for many implant users; (3) Perception of timbre, which is usually evaluated by experimental procedures that require subjects to identify musical instrument sounds, is generally unsatisfactory; (4) Implant users tend to rate the quality of musical sounds as less pleasant than listeners with normal hearing; (5) Auditory training programs that have been devised specifically to provide implant users with structured musical listening experience may improve the subjective acceptability of music that is heard through a prosthesis; (6) Pitch perception might be improved by designing innovative sound processors that use both temporal and spatial patterns of electric stimulation more effectively and precisely to overcome the inherent limitations of signal coding in existing implant systems; (7) For the growing population of implant recipients who have usable acoustic hearing, at least for low-frequency sounds, perception of music is likely to be much better with combined acoustic and electric stimulation than is typical for deaf people who rely solely on the hearing provided by their prostheses.