Modulation enhancement in the electrical signal improves perception of interaural time differences with bimodal stimulation

Interaural Place-of-Stimulation Mismatch Estimates Using CT Scans and Binaural Perception, But Not Pitch, Are Consistent in Cochlear-Implant Users

Bilateral cochlear implants (BI-CIs) or a CI for single-sided deafness (SSD-CI; one normally functioning acoustic ear) can partially restore spatial-hearing abilities, including sound localization and speech understanding in noise. For these populations, however, interaural place-of-stimulation mismatch can occur and thus diminish binaural sensitivity that relies on interaurally frequency-matched neurons. This study examined whether plasticity-reorganization of central neural pathways over time-can compensate for peripheral interaural place mismatch. We hypothesized differential plasticity across two systems: none for binaural processing but adaptation for pitch perception toward frequencies delivered by the specific electrodes. Interaural place mismatch was evaluated in 19 BI-CI and 23 SSD-CI human subjects (both sexes) using binaural processing (interaural-time-difference discrimination with simultaneous bilateral stimulation), pitch perception (pitch ranking for single electrodes or acoustic tones with sequential bilateral stimulation), and physical electrode-location estimates from computed-tomography (CT) scans. On average, CT scans revealed relatively little BI-CI interaural place mismatch (26° insertion-angle mismatch) but a relatively large SSD-CI mismatch, particularly at low frequencies (166° for an electrode tuned to 300 Hz, decreasing to 14° at 7000 Hz). For BI-CI subjects, the three metrics were in agreement because there was little mismatch. For SSD-CI subjects, binaural and CT measurements were in agreement, suggesting little binaural-system plasticity induced by mismatch. The pitch measurements disagreed with binaural and CT measurements, suggesting place-pitch plasticity or a procedural bias. These results suggest that reducing interaural place mismatch and potentially improving binaural processing by reprogramming the CI frequency allocation would be better done using CT-scan than pitch information.SIGNIFICANCE STATEMENT Electrode-array placement for cochlear implants (bionic prostheses that partially restore hearing) does not explicitly align neural representations of frequency information. The resulting interaural place-of-stimulation mismatch can diminish spatial-hearing abilities. In this study, adults with two cochlear implants showed reasonable interaural alignment, whereas those with one cochlear implant but normal hearing in the other ear often showed mismatch. In cases of mismatch, binaural sensitivity was best when the same cochlear locations were stimulated in both ears, suggesting that binaural brainstem pathways do not experience plasticity to compensate for mismatch. In contrast, interaurally pitch-matched electrodes deviated from cochlear-location estimates and did not optimize binaural sensitivity. Clinical correction of interaural place mismatch using binaural or computed-tomography (but not pitch) information may improve spatial-hearing benefits.

Reweighting of Binaural Localization Cues in Bilateral Cochlear-Implant Listeners

Normal-hearing (NH) listeners rely on two binaural cues, the interaural time (ITD) and level difference (ILD), for azimuthal sound localization. Cochlear-implant (CI) listeners, however, rely almost entirely on ILDs. One reason is that present-day clinical CI stimulation strategies do not convey salient ITD cues. But even when presenting ITDs under optimal conditions using a research interface, ITD sensitivity is lower in CI compared to NH listeners. Since it has recently been shown that NH listeners change their ITD/ILD weighting when only one of the cues is consistent with visual information, such reweighting might add to CI listeners’ low perceptual contribution of ITDs, given their daily exposure to reliable ILDs but unreliable ITDs. Six bilateral CI listeners completed a multi-day lateralization training visually reinforcing ITDs, flanked by a pre- and post-measurement of ITD/ILD weights without visual reinforcement. Using direct electric stimulation, we presented 100- and 300-pps pulse trains at a single interaurally place-matched electrode pair, conveying ITDs and ILDs in various spatially consistent and inconsistent combinations. The listeners’ task was to lateralize the stimuli in a virtual environment. Additionally, ITD and ILD thresholds were measured before and after training. For 100-pps stimuli, the lateralization training increased the contribution of ITDs slightly, but significantly. Thresholds were neither affected by the training nor correlated with weights. For 300-pps stimuli, ITD weights were lower and ITD thresholds larger, but there was no effect of training. On average across test sessions, adding azimuth-dependent ITDs to stimuli containing ILDs increased the extent of lateralization for both 100- and 300-pps stimuli. The results suggest that low-rate ITD cues, robustly encoded with future CI systems, may be better exploitable for sound localization after increasing their perceptual weight via training.

High-resolution temporal weighting of interaural time differences in speech

Previous studies have shown that for high-rate click trains and low-frequency pure tones, interaural time differences (ITDs) at the onset of stimulus contribute most strongly to the overall lateralization percept (receive the largest perceptual weight). Previous studies have also shown that when these stimuli are modulated, ITDs during the rising portion of the modulation cycle receive increased perceptual weight. Baltzell, Cho, Swaminathan, and Best [(2020). J. Acoust. Soc. Am. 147, 3883-3894] measured perceptual weights for a pair of spoken words ("two" and "eight"), and found that word-initial phonemes receive larger weight than word-final phonemes, suggesting a "word-onset dominance" for speech. Generalizability of this conclusion was limited by a coarse temporal resolution and limited stimulus set. In the present study, temporal weighting functions (TWFs) were measured for four spoken words ("two," "eight," "six," and "nine"). Stimuli were partitioned into 30-ms bins, ITDs were applied independently to each bin, and lateralization judgements were obtained. TWFs were derived using a hierarchical regression model. Results suggest that "word-initial" onset dominance does not generalize across words and that TWFs depend in part on acoustic changes throughout the stimulus. Two model-based predictions were generated to account for observed TWFs, but neither could fully account for the perceptual data.

Speech Understanding With Bimodal Stimulation Is Determined by Monaural Signal to Noise Ratios: No Binaural Cue Processing Involved

OBJECTIVES

To investigate the mechanisms behind binaural and spatial effects in speech understanding for bimodal cochlear implant listeners. In particular, to test our hypothesis that their speech understanding can be characterized by means of monaural signal to noise ratios, rather than complex binaural cue processing such as binaural unmasking.

DESIGN

We applied a semantic framework to characterize binaural and spatial effects in speech understanding on an extensive selection of the literature on bimodal listeners. In addition, we performed two experiments in which we measured speech understanding in different masker types (1) using head-related transfer functions, and (2) while adapting the broadband signal to noise ratios in both ears independently. We simulated bimodal hearing with a vocoder in one ear (the cochlear implant side) and a low-pass filter in the other ear (the hearing aid side). By design, the cochlear implant side was the main contributor to speech understanding in our simulation.

RESULTS

We found that spatial release from masking can be explained as a simple trade-off between a monaural change in signal to noise at the cochlear implant side (quantified as the head shadow effect) and an opposite change in signal to noise at the hearing aid side (quantified as a change in bimodal benefit). In simulated bimodal listeners, we found that for every 1 dB increase in signal to noise ratio at the hearing aid side, the bimodal benefit improved by approximately 0.4 dB in signal to noise ratio.

CONCLUSIONS

Although complex binaural cue processing is often implicated when discussing speech intelligibility in adverse listening conditions, performance can simply be explained based on monaural signal to noise ratios for bimodal listeners.

Spatial Release From Informational and Energetic Masking in Bimodal and Bilateral Cochlear Implant Users

Purpose Spatially separating speech and background noise improves speech understanding in normal-hearing listeners, an effect referred to as spatial release from masking (SRM). In cochlear implant (CI) users, SRM has often been demonstrated using asymmetric noise configurations, which maximize benefit from head shadow and the potential availability of binaural cues. In contrast, SRM in symmetrical configurations has been minimal to absent in CI users. We examined the interaction between two types of maskers (informational and energetic) and SRM in bimodal and bilateral CI users. We hypothesized that SRM would be absent or "negative" using symmetrically separated noise maskers. Second, we hypothesized that bimodal listeners would exhibit greater release from informational masking due to access to acoustic information in the non-CI ear. Method Participants included 10 bimodal and 10 bilateral CI users. Speech understanding in noise was tested in 24 conditions: 3 spatial configurations (S₀N₀, S₀N_45&315, S₀N_90&270) × 2 masker types (speech, signal-correlated noise) × 2 listening configurations (best-aided, CI-alone) × 2 talker gender conditions (different-gender, same-gender). Results In support of our first hypothesis, both groups exhibited negative SRM with increasing spatial separation. In opposition to our second hypothesis, both groups exhibited similar magnitudes of release from informational masking. The magnitude of release was greater for bimodal listeners, though this difference failed to reach statistical significance. Conclusions Both bimodal and bilateral CI recipients exhibited negative SRM. This finding is consistent with CI signal processing limitations, the audiologic factors associated with SRM, and known effects of behind-the-ear microphone technology. Though release from informational masking was not significantly different across groups, the magnitude of release was greater for bimodal listeners. This suggests that bimodal listeners may be at least marginally more susceptible to informational masking than bilateral CI users, though further research is warranted.

Development of cortical auditory responses to speech in noise in unilaterally deaf adults following cochlear implantation

BACKGROUND

For patients with single-sided deafness (SSD), restoration of binaural function via cochlear implant (CI) has been shown to improve speech understanding in noise. The objective of this study was to investigate changes in behavioral performance and cortical auditory responses following cochlear implantation.

DESIGN

Prospective longitudinal study.

SETTING

Tertiary referral center.

METHODS

Six adults with SSD were tested before and 12 months post-activation of the CI. Six normal hearing (NH) participants served as experimental controls. Speech understanding in noise was evaluated for various spatial conditions. Cortical auditory evoked potentials were recorded with /ba/ stimuli in quiet and in noise. Global field power and responses at Cz were analyzed.

RESULTS

Speech understanding in noise significantly improved with the CI when speech was presented to the CI ear and noise to the normal ear (p<0.05), but remained poorer than that of NH controls (p<0.05). N1 peak amplitude measure in noise significantly increased after CI activation (p<0.05), but remained lower than that of NH controls (p<0.05) at 12 months. After 12 months of CI experience, cortical responses in noise became more comparable between groups.

CONCLUSION

Binaural restoration in SSD patients via cochlear implantation improved speech performance noise and cortical responses. While behavioral performance and cortical auditory responses improved, SSD-CI outcomes remained poorer than that of NH controls in most cases, suggesting only partial restoration of binaural hearing.

Temporal quantization deteriorates the discrimination of interaural time differences

Cochlear implants (CIs) often replace acoustic temporal fine structure by a fixed-rate pulse train. If the pulse timing is arbitrary (that is, not based on the phase information of the acoustic signal), temporal information is quantized by the pulse period. This temporal quantization is probably imperceptible with current clinical devices. However, it could result in large temporal jitter for strategies that aim to improve bilateral and bimodal CI users' perception of interaural time differences (ITDs), such as envelope enhancement. In an experiment with 16 normal-hearing listeners, it is shown that such jitter could deteriorate ITD perception for temporal quantization that corresponds to the often-used stimulation rate of 900 pulses per second (pps): the just-noticeable difference in ITD with quantization was 177 μs as compared to 129 μs without quantization. For smaller quantization step sizes, no significant deterioration of ITD perception was found. In conclusion, the binaural system can only average out the effect of temporal quantization to some extent, such that pulse timing should be well-considered. As this psychophysical procedure was somewhat unconventional, different procedural parameters were compared by simulating a number of commonly used two-down one-up adaptive procedures in Appendix B.

Improving Interaural Time Difference Sensitivity Using Short Inter-pulse Intervals with Amplitude-Modulated Pulse Trains in Bilateral Cochlear Implants

Interaural time differences (ITDs) at low frequencies are important for sound localization and spatial speech unmasking. These ITD cues are not encoded in commonly used envelope-based stimulation strategies for cochlear implants (CIs) using high pulse rates. However, ITD sensitivity can be improved by adding extra pulses with short inter-pulse intervals (SIPIs) in unmodulated high-rate trains. Here, we investigated whether this improvement also applies to amplitude-modulated (AM) high-rate pulse trains. To this end, we systematically varied the temporal position of SIPI pulses within the envelope cycle (SIPI phase), the fundamental frequency (F0) of AM (125 Hz and 250 Hz), and AM depth (from 0.1 to 0.9). Stimuli were presented at an interaurally place-matched electrode pair at a reference pulse rate of 1000 pulses/s. Participants performed an ITD-based left/right discrimination task. SIPI insertion resulted in improved ITD sensitivity throughout the range of modulation depths and for both male and female F0s. The improvements were largest for insertion at and around the envelope peak. These results are promising for conveying salient ITD cues at high pulse rates commonly used to encode speech information.

Bimodal Cochlear Implant Listeners' Ability to Perceive Minimal Audible Angle Differences

BACKGROUND

Bilateral inputs should ideally improve sound localization and speech understanding in noise. However, for many bimodal listeners [i.e., individuals using a cochlear implant (CI) with a contralateral hearing aid (HA)], such bilateral benefits are at best, inconsistent. The degree to which clinically available HA and CI devices can function together to preserve interaural time and level differences (ITDs and ILDs, respectively) enough to support the localization of sound sources is a question with important ramifications for speech understanding in complex acoustic environments.

PURPOSE

To determine if bimodal listeners are sensitive to changes in spatial location in a minimum audible angle (MAA) task.

RESEARCH DESIGN

Repeated-measures design.

STUDY SAMPLE

Seven adult bimodal CI users (28-62 years). All listeners reported regular use of digital HA technology in the nonimplanted ear.

DATA COLLECTION AND ANALYSIS

Seven bimodal listeners were asked to balance the loudness of prerecorded single syllable utterances. The loudness-balanced stimuli were then presented via direct audio inputs of the two devices with an ITD applied. The task of the listener was to determine the perceived difference in processing delay (the interdevice delay [IDD]) between the CI and HA devices. Finally, virtual free-field MAA performance was measured for different spatial locations both with and without inclusion of the IDD correction, which was added with the intent to perceptually synchronize the devices.

RESULTS

During the loudness-balancing task, all listeners required increased acoustic input to the HA relative to the CI most comfortable level to achieve equal interaural loudness. During the ITD task, three listeners could perceive changes in intracranial position by distinguishing sounds coming from the left or from the right hemifield; when the CI was delayed by 0.73, 0.67, or 1.7 msec, the signal lateralized from one side to the other. When MAA localization performance was assessed, only three of the seven listeners consistently achieved above-chance performance, even when an IDD correction was included. It is not clear whether the listeners who were able to consistently complete the MAA task did so via binaural comparison or by extracting monaural loudness cues. Four listeners could not perform the MAA task, even though they could have used a monaural loudness cue strategy.

CONCLUSIONS

These data suggest that sound localization is extremely difficult for most bimodal listeners. This difficulty does not seem to be caused by large loudness imbalances and IDDs. Sound localization is best when performed via a binaural comparison, where frequency-matched inputs convey ITD and ILD information. Although low-frequency acoustic amplification with a HA when combined with a CI may produce an overlapping region of frequency-matched inputs and thus provide an opportunity for binaural comparisons for some bimodal listeners, our study showed that this may not be beneficial or useful for spatial location discrimination tasks. The inability of our listeners to use monaural-level cues to perform the MAA task highlights the difficulty of using a HA and CI together to glean information on the direction of a sound source.

Interaural Time-Difference Discrimination as a Measure of Place of Stimulation for Cochlear-Implant Users With Single-Sided Deafness

Current clinical practice in programming a cochlear implant (CI) for individuals with single-sided deafness (SSD) is to maximize the transmission of speech information via the implant, with the implicit assumption that this will also result in improved spatial-hearing abilities. However, binaural sensitivity is reduced by interaural place-of-stimulation mismatch, a likely occurrence with a standard CI frequency-to-electrode allocation table (FAT). As a step toward reducing interaural mismatch, this study investigated whether a test of interaural-time-difference (ITD) discrimination could be used to estimate the acoustic frequency yielding the best place match for a given CI electrode. ITD-discrimination performance was measured by presenting 300-ms bursts of 100-pulses-per-second electrical pulse trains to a single CI electrode and band-limited pulse trains with variable carrier frequencies to the acoustic ear. Listeners discriminated between two reference intervals (four bursts each with constant ITD) and a moving target interval (four bursts with variable ITD). For 17 out of the 26 electrodes tested across eight listeners, the function describing the relationship between ITD-discrimination performance and carrier frequency had a discernable peak where listeners achieved 70% to 100% performance. On average, this peak occurred 1.15 octaves above the CI manufacturer’s default FAT. ITD discrimination shows promise as a method of estimating the cochlear place of stimulation for a given electrode, thereby providing information to optimize the FAT for SSD-CI listeners.

Interaural Time Difference Perception with a Cochlear Implant and a Normal Ear

Currently there is a growing population of cochlear-implant (CI) users with (near) normal hearing in the non-implanted ear. This configuration is often called SSD (single-sided deafness) CI. The goal of the CI is often to improve spatial perception, so the question raises to what extent SSD CI listeners are sensitive to interaural time differences (ITDs). In a controlled lab setup, sensitivity to ITDs was investigated in 11 SSD CI listeners. The stimuli were 100-pps pulse trains on the CI side and band-limited click trains on the acoustic side. After determining level balance and the delay needed to achieve synchronous stimulation of the two ears, the just noticeable difference in ITD was measured using an adaptive procedure. Seven out of 11 listeners were sensitive to ITDs, with a median just noticeable difference of 438 μs. Out of the four listeners who were not sensitive to ITD, one listener reported binaural fusion, and three listeners reported no binaural fusion. To enable ITD sensitivity, a frequency-dependent delay of the electrical stimulus was required to synchronize the electric and acoustic signals at the level of the auditory nerve. Using subjective fusion measures and refined by ITD sensitivity, it was possible to match a CI electrode to an acoustic frequency range. This shows the feasibility of these measures for the allocation of acoustic frequency ranges to electrodes when fitting a CI to a subject with (near) normal hearing in the contralateral ear.

Activity in Human Auditory Cortex Represents Spatial Separation Between Concurrent Sounds

The primary and posterior auditory cortex (AC) are known for their sensitivity to spatial information, but how this information is processed is not yet understood. AC that is sensitive to spatial manipulations is also modulated by the number of auditory streams present in a scene (Smith et al., 2010), suggesting that spatial and nonspatial cues are integrated for stream segregation. We reasoned that, if this is the case, then it is the distance between sounds rather than their absolute positions that is essential. To test this hypothesis, we measured human brain activity in response to spatially separated concurrent sounds with fMRI at 7 tesla in five men and five women. Stimuli were spatialized amplitude-modulated broadband noises recorded for each participant via in-ear microphones before scanning. Using a linear support vector machine classifier, we investigated whether sound location and/or location plus spatial separation between sounds could be decoded from the activity in Heschl's gyrus and the planum temporale. The classifier was successful only when comparing patterns associated with the conditions that had the largest difference in perceptual spatial separation. Our pattern of results suggests that the representation of spatial separation is not merely the combination of single locations, but rather is an independent feature of the auditory scene.SIGNIFICANCE STATEMENT Often, when we think of auditory spatial information, we think of where sounds are coming from-that is, the process of localization. However, this information can also be used in scene analysis, the process of grouping and segregating features of a soundwave into objects. Essentially, when sounds are further apart, they are more likely to be segregated into separate streams. Here, we provide evidence that activity in the human auditory cortex represents the spatial separation between sounds rather than their absolute locations, indicating that scene analysis and localization processes may be independent.

Models of the electrically stimulated binaural system: A review

In an increasing number of countries, the standard treatment for deaf individuals is moving toward the implantation of two cochlear implants. Today's device technology and fitting procedure, however, appears as if the two implants would serve two independent ears and brains. Many experimental studies have demonstrated that after careful matching and balancing of left and right stimulation in controlled laboratory studies most patients have almost normal sensitivity to interaural level differences and some sensitivity to interaural time differences (ITDs). Mechanisms underlying the limited ITD sensitivity are still poorly understood and many different aspects may contribute. Recent pioneering computational approaches identified some of the functional implications the electric input imposes on the neural brainstem circuits. Simultaneously these studies have raised new questions and certainly demonstrated that further refinement of the model stages is necessary. They join the experimental study's conclusions that binaural device technology, binaural fitting, specific speech coding strategies, and binaural signal processing algorithms are obviously missing components to maximize the benefit of bilateral implantation. Within this review, the existing models of the electrically stimulated binaural system are explained, compared, and discussed from a viewpoint of a "CI device with auditory system" and from that of neurophysiological research.

A method to enhance the use of interaural time differences for cochlear implants in reverberant environments

The ability of normal-hearing (NH) listeners to exploit interaural time difference (ITD) cues conveyed in the modulated envelopes of high-frequency sounds is poor compared to ITD cues transmitted in the temporal fine structure at low frequencies. Sensitivity to envelope ITDs is further degraded when envelopes become less steep, when modulation depth is reduced, and when envelopes become less similar between the ears, common factors when listening in reverberant environments. The vulnerability of envelope ITDs is particularly problematic for cochlear implant (CI) users, as they rely on information conveyed by slowly varying amplitude envelopes. Here, an approach to improve access to envelope ITDs for CIs is described in which, rather than attempting to reduce reverberation, the perceptual saliency of cues relating to the source is increased by selectively sharpening peaks in the amplitude envelope judged to contain reliable ITDs. Performance of the algorithm with room reverberation was assessed through simulating listening with bilateral CIs in headphone experiments with NH listeners. Relative to simulated standard CI processing, stimuli processed with the algorithm generated lower ITD discrimination thresholds and increased extents of laterality. Depending on parameterization, intelligibility was unchanged or somewhat reduced. The algorithm has the potential to improve spatial listening with CIs.