Sensitivity to Envelope Interaural Time Differences at High Modulation Rates

Rate dependent neural responses of interaural-time-difference cues in fine-structure and envelope

Advancements in cochlear implants (CIs) have led to a significant increase in bilateral CI users, especially among children. Yet, most bilateral CI users do not fully achieve the intended binaural benefit due to potential limitations in signal processing and/or surgical implant positioning. One crucial auditory cue that normal hearing (NH) listeners can benefit from is the interaural time difference (ITD), i.e., the time difference between the arrival of a sound at two ears. The ITD sensitivity is thought to be heavily relying on the effective utilization of temporal fine structure (very rapid oscillations in sound). Unfortunately, most current CIs do not transmit such true fine structure. Nevertheless, bilateral CI users have demonstrated sensitivity to ITD cues delivered through envelope or interaural pulse time differences, i.e., the time gap between the pulses delivered to the two implants. However, their ITD sensitivity is significantly poorer compared to NH individuals, and it further degrades at higher CI stimulation rates, especially when the rate exceeds 300 pulse per second. The overall purpose of this research thread is to improve spatial hearing abilities in bilateral CI users. This study aims to develop electroencephalography (EEG) paradigms that can be used with clinical settings to assess and optimize the delivery of ITD cues, which are crucial for spatial hearing in everyday life. The research objective of this article was to determine the effect of CI stimulation pulse rate on the ITD sensitivity, and to characterize the rate-dependent degradation in ITD perception using EEG measures. To develop protocols for bilateral CI studies, EEG responses were obtained from NH listeners using sinusoidal-amplitude-modulated (SAM) tones and filtered clicks with changes in either fine structure ITD (ITDFS) or envelope ITD (ITDENV). Multiple EEG responses were analyzed, which included the subcortical auditory steady-state responses (ASSRs) and cortical auditory evoked potentials (CAEPs) elicited by stimuli onset, offset, and changes. Results indicated that acoustic change complex (ACC) responses elicited by ITDENV changes were significantly smaller or absent compared to those elicited by ITDFS changes. The ACC morphologies evoked by ITDFS changes were similar to onset and offset CAEPs, although the peak latencies were longest for ACC responses and shortest for offset CAEPs. The high-frequency stimuli clearly elicited subcortical ASSRs, but smaller than those evoked by lower carrier frequency SAM tones. The 40-Hz ASSRs decreased with increasing carrier frequencies. Filtered clicks elicited larger ASSRs compared to high-frequency SAM tones, with the order being 40 > 160 > 80> 320 Hz ASSR for both stimulus types. Wavelet analysis revealed a clear interaction between detectable transient CAEPs and 40-Hz ASSRs in the time-frequency domain for SAM tones with a low carrier frequency.

Review of Binaural Processing With Asymmetrical Hearing Outcomes in Patients With Bilateral Cochlear Implants

Bilateral cochlear implants (BiCIs) result in several benefits, including improvements in speech understanding in noise and sound source localization. However, the benefit bilateral implants provide among recipients varies considerably across individuals. Here we consider one of the reasons for this variability: difference in hearing function between the two ears, that is, interaural asymmetry. Thus far, investigations of interaural asymmetry have been highly specialized within various research areas. The goal of this review is to integrate these studies in one place, motivating future research in the area of interaural asymmetry. We first consider bottom-up processing, where binaural cues are represented using excitation-inhibition of signals from the left ear and right ear, varying with the location of the sound in space, and represented by the lateral superior olive in the auditory brainstem. We then consider top-down processing via predictive coding, which assumes that perception stems from expectations based on context and prior sensory experience, represented by cascading series of cortical circuits. An internal, perceptual model is maintained and updated in light of incoming sensory input. Together, we hope that this amalgamation of physiological, behavioral, and modeling studies will help bridge gaps in the field of binaural hearing and promote a clearer understanding of the implications of interaural asymmetry for future research on optimal patient interventions.

The role of carrier spectral composition in the perception of musical pitch

Temporal envelope fluctuations of natural sounds convey critical information to speech and music processing. In particular, musical pitch perception is assumed to be primarily underlined by temporal envelope encoding. While increasing evidence demonstrates the importance of carrier fine structure to complex pitch perception, how carrier spectral information affects musical pitch perception is less clear. Here, transposed tones designed to convey identical envelope information across different carriers were used to assess the effects of carrier spectral composition to pitch discrimination and musical-interval and melody identifications. Results showed that pitch discrimination thresholds became lower (better) with increasing carrier frequencies from 1k to 10k Hz, with performance comparable to that of pure sinusoids. Musical interval and melody defined by the periodicity of sine- or harmonic complex envelopes across carriers were identified with greater than 85% accuracy even on a 10k-Hz carrier. Moreover, enhanced interval and melody identification performance was observed with increasing carrier frequency up to 6k Hz. Findings suggest a perceptual enhancement of temporal envelope information with increasing carrier spectral region in musical pitch processing, at least for frequencies up to 6k Hz. For carriers in the extended high-frequency region (8-20k Hz), the use of temporal envelope information to music pitch processing may vary depending on task requirement. Collectively, these results implicate the fidelity of temporal envelope information to musical pitch perception is more pronounced than previously considered, with ecological implications.

The Effects of Middle-ear Stiffness on the Auditory Brainstem Neural Encoding of Phase

The middle-ear system relies on a balance of mass and stiffness characteristics for transmitting sound from the external environment to the cochlea and auditory neural pathway. Phase is one aspect of sound that, when transmitted and encoded by both ears, contributes to binaural cue sensitivity and spatial hearing. The study aims were (i) to investigate the effects of middle-ear stiffness on the auditory brainstem neural encoding of phase in human adults with normal pure-tone thresholds and (ii) to investigate the relationships between middle-ear stiffness-induced changes in wideband acoustic immittance and neural encoding of phase. The auditory brainstem neural encoding of phase was measured using the auditory steady-state response (ASSR) with and without middle-ear stiffness elicited via contralateral activation of the middle-ear muscle reflex (MEMR). Middle-ear stiffness was quantified using a wideband acoustic immittance assay of acoustic absorbance. Statistical analyses demonstrated decreased ASSR phase lag and decreased acoustic absorbance with contralateral activation of the MEMR, consistent with increased middle-ear stiffness changing the auditory brainstem neural encoding of phase. There were no statistically significant correlations between stiffness-induced changes in wideband acoustic absorbance and ASSR phase. The findings of this study may have important implications for understanding binaural cue sensitivity and horizontal plane sound localization in audiologic and otologic clinical populations that demonstrate changes in middle-ear stiffness, including cochlear implant recipients who use combined electric and binaural acoustic hearing and otosclerosis patients.

Asymmetric temporal envelope sensitivity: Within- and across-ear envelope comparisons in listeners with bilateral cochlear implants

For listeners with bilateral cochlear implants (BiCIs), patient-specific differences in the interface between cochlear implant (CI) electrodes and the auditory nerve can lead to degraded temporal envelope information, compromising the ability to distinguish between targets of interest and background noise. It is unclear how comparisons of degraded temporal envelope information across spectral channels (i.e., electrodes) affect the ability to detect differences in the temporal envelope, specifically amplitude modulation (AM) rate. In this study, two pulse trains were presented simultaneously via pairs of electrodes in different places of stimulation, within and/or across ears, with identical or differing AM rates. Results from 11 adults with BiCIs indicated that sensitivity to differences in AM rate was greatest when stimuli were paired between different places of stimulation in the same ear. Sensitivity from pairs of electrodes was predicted by the poorer electrode in the pair or the difference in fidelity between both electrodes in the pair. These findings suggest that electrodes yielding poorer temporal fidelity act as a bottleneck to comparisons of temporal information across frequency and ears, limiting access to the cues used to segregate sounds, which has important implications for device programming and optimizing patient outcomes with CIs.

Simulation of ITD-Dependent Single-Neuron Responses Under Electrical Stimulation and with Amplitude-Modulated Acoustic Stimuli

Interaural time difference (ITD) sensitivity with cochlear implant stimulation is remarkably similar to envelope ITD sensitivity using conventional acoustic stimulation. This holds true for human perception, as well as for neural response rates recorded in the inferior colliculus of several mammalian species. We hypothesize that robust excitatory-inhibitory (EI) interaction is the dominant mechanism. Therefore, we connected the same single EI-model neuron to either a model of the normal acoustic auditory periphery or to a model of the electrically stimulated auditory nerve. The model captured most features of the experimentally obtained response properties with electric stimulation, such as the shape of rate-ITD functions, the dependence on stimulation level, and the pulse rate or modulation-frequency dependence. Rate-ITD functions with high-rate, amplitude-modulated electric stimuli were very similar to their acoustic counterparts. Responses obtained with unmodulated electric pulse trains most resembled acoustic filtered clicks. The fairly rapid decline of ITD sensitivity at rates above 300 pulses or cycles per second is correctly simulated by the 3.1-ms time constant of the inhibitory post-synaptic conductance. As the model accounts for these basic properties, it is expected to help in understanding and quantifying the binaural hearing abilities with electric stimulation when integrated in bigger simulation frameworks.

Sensitivity to Envelope Interaural Time Differences: Modeling Auditory Modulation Filtering

For amplitude-modulated sound, the envelope interaural time difference (ITD_ENV) is a potential cue for sound-source location. ITD_ENV is encoded in the lateral superior olive (LSO) of the auditory brainstem, by excitatory-inhibitory (EI) neurons receiving ipsilateral excitation and contralateral inhibition. Between human listeners, sensitivity to ITD_ENV varies considerably, but ultimately decreases with increasing stimulus carrier frequency, and decreases more strongly with increasing modulation rate. Mechanisms underlying the variation in behavioral sensitivity remain unclear. Here, with increasing carrier frequency (4-10 kHz), as we phenomenologically model the associated decrease in ITD_ENV sensitivity using arbitrarily fewer neurons consistent across populations, we computationally model the variable sensitivity across human listeners and modulation rates (32-800 Hz) as the decreasing range of membrane frequency responses in LSO neurons. Transposed tones stimulate a bilateral auditory-periphery model, driving model EI neurons where electrical membrane impedance filters the frequency content of inputs driven by amplitude-modulated sound, evoking modulation filtering. Calculated from Fisher information in spike-rate functions of ITD_ENV, for model EI neuronal populations distinctly reflecting the LSO range in membrane frequency responses, just-noticeable differences in ITD_ENV collectively reproduce the largest variation in ITD_ENV sensitivity across human listeners. These slow to fast model populations each generally match the best human ITD_ENV sensitivity at a progressively higher modulation rate, by membrane-filtering and spike-generation properties producing realistically less than Poisson variance. Non-resonant model EI neurons are also sensitive to interaural intensity differences. With peripheral filters centered between carrier frequency and modulation sideband, fast resonant model EI neurons extend ITD_ENV sensitivity above 500-Hz modulation.

Sound Localization in Single-Sided Deaf Participants Provided With a Cochlear Implant

Spatial hearing is crucial in real life but deteriorates in participants with severe sensorineural hearing loss or single-sided deafness. This ability can potentially be improved with a unilateral cochlear implant (CI). The present study investigated measures of sound localization in participants with single-sided deafness provided with a CI. Sound localization was measured separately at eight loudspeaker positions (4°, 30°, 60°, and 90°) on the CI side and on the normal-hearing side. Low- and high-frequency noise bursts were used in the tests to investigate possible differences in the processing of interaural time and level differences. Data were compared to normal-hearing adults aged between 20 and 83. In addition, the benefit of the CI in speech understanding in noise was compared to the localization ability. Fifteen out of 18 participants were able to localize signals on the CI side and on the normal-hearing side, although performance was highly variable across participants. Three participants always pointed to the normal-hearing side, irrespective of the location of the signal. The comparison with control data showed that participants had particular difficulties localizing sounds at frontal locations and on the CI side. In contrast to most previous results, participants were able to localize low-frequency signals, although they localized high-frequency signals more accurately. Speech understanding in noise was better with the CI compared to testing without CI, but only at a position where the CI also improved sound localization. Our data suggest that a CI can, to a large extent, restore localization in participants with single-sided deafness. Difficulties may remain at frontal locations and on the CI side. However, speech understanding in noise improves when wearing the CI. The treatment with a CI in these participants might provide real-world benefits, such as improved orientation in traffic and speech understanding in difficult listening situations.

Neural rate difference model can account for lateralization of high-frequency stimuli

Lateralization of complex high-frequency sounds is conveyed by interaural level differences (ILDs) and interaural time differences (ITDs) in the envelope. In this work, the authors constructed an auditory model and simulate data from three previous behavioral studies obtained with, in total, over 1000 different amplitude-modulated stimuli. The authors combine a well-established auditory periphery model with a functional count-comparison model for binaural excitatory-inhibitory (EI) interaction. After parameter optimization of the EI-model stage, the hemispheric rate-difference between pairs of EI-model neurons relates linearly with the extent of laterality in human listeners. If a certain ILD and a certain envelope ITD each cause a similar extent of laterality, they also produce a similar rate difference in the same model neurons. After parameter optimization, the model accounts for 95.7% of the variance in the largest dataset, in which amplitude modulation depth, rate of modulation, modulation exponent, ILD, and envelope ITD were varied. The model also accounts for 83% of the variances in each of the other two datasets using the same EI model parameters.

Temporal quantization deteriorates the discrimination of interaural time differences

Cochlear implants (CIs) often replace acoustic temporal fine structure by a fixed-rate pulse train. If the pulse timing is arbitrary (that is, not based on the phase information of the acoustic signal), temporal information is quantized by the pulse period. This temporal quantization is probably imperceptible with current clinical devices. However, it could result in large temporal jitter for strategies that aim to improve bilateral and bimodal CI users' perception of interaural time differences (ITDs), such as envelope enhancement. In an experiment with 16 normal-hearing listeners, it is shown that such jitter could deteriorate ITD perception for temporal quantization that corresponds to the often-used stimulation rate of 900 pulses per second (pps): the just-noticeable difference in ITD with quantization was 177 μs as compared to 129 μs without quantization. For smaller quantization step sizes, no significant deterioration of ITD perception was found. In conclusion, the binaural system can only average out the effect of temporal quantization to some extent, such that pulse timing should be well-considered. As this psychophysical procedure was somewhat unconventional, different procedural parameters were compared by simulating a number of commonly used two-down one-up adaptive procedures in Appendix B.

Effects of rate and age in processing interaural time and level differences in normal-hearing and bilateral cochlear-implant listeners

Bilateral cochlear implants (BICIs) provide improved sound localization and speech understanding in noise compared to unilateral CIs. However, normal-hearing (NH) listeners demonstrate superior binaural processing abilities compared to BICI listeners. This investigation sought to understand differences between NH and BICI listeners' processing of interaural time differences (ITDs) and interaural level differences (ILDs) as a function of fine-structure and envelope rate using an intracranial lateralization task. The NH listeners were presented band-limited acoustical pulse trains and sinusoidally amplitude-modulated tones using headphones, and the BICI listeners were presented single-electrode electrical pulse trains using direct stimulation. Lateralization range increased as fine-structure rate increased for ILDs in BICI listeners. Lateralization range decreased for rates above 100 Hz for fine-structure ITDs, but decreased for rates lower or higher than 100 Hz for envelope ITDs in both groups. Lateralization ranges for ITDs were smaller for BICI listeners on average. After controlling for age, older listeners showed smaller lateralization ranges and BICI listeners had a more rapid decline for ITD sensitivity at 300 pulses per second. This work suggests that age confounds comparisons between NH and BICI listeners in temporal processing tasks and that some NH-BICI binaural processing differences persist even when age differences are adequately addressed.

Effects of age on sensitivity to interaural time differences in envelope and fine structure, individually and in combination

Sensitivity to interaural time differences (ITDs) in envelope and temporal fine structure (TFS) of amplitude-modulated (AM) tones was assessed for young and older subjects, all with clinically normal hearing at the carrier frequencies of 250 and 500 Hz. Some subjects had hearing loss at higher frequencies. In experiment 1, thresholds for detecting changes in ITD were measured when the ITD was present in the TFS alone (ITD_TFS), the envelope alone (ITD_ENV), or both (ITD_TFS/ENV). Thresholds tended to be higher for the older than for the young subjects. ITD_ENV thresholds were much higher than ITD_TFS thresholds, while ITD_TFS/ENV thresholds were similar to ITD_TFS thresholds. ITD_TFS thresholds were lower than ITD thresholds obtained with an unmodulated pure tone, indicating that uninformative AM can improve ITD_TFS discrimination. In experiment 2, equally detectable values of ITD_TFS and ITD_ENV were combined so as to give consistent or inconsistent lateralization. There were large individual differences, but several subjects gave scores that were much higher than would be expected from the optimal combination of independent sources of information, even for the inconsistent condition. It is suggested that ITD_TFS and ITD_ENV cues are processed partly independently, but that both cues influence lateralization judgments, even when one cue is uninformative.

Sound source localization identification accuracy: Envelope dependencies

Sound source localization accuracy as measured in an identification procedure in a front azimuth sound field was studied for click trains, modulated noises, and a modulated tonal carrier. Sound source localization accuracy was determined as a function of the number of clicks in a 64 Hz click train and click rate for a 500 ms duration click train. The clicks were either broadband or high-pass filtered. Sound source localization accuracy was also measured for a single broadband filtered click and compared to a similar broadband filtered, short-duration noise. Sound source localization accuracy was determined as a function of sinusoidal amplitude modulation and the "transposed" process of modulation of filtered noises and a 4 kHz tone. Different rates (16 to 512 Hz) of modulation (including unmodulated conditions) were used. Providing modulation for filtered click stimuli, filtered noises, and the 4 kHz tone had, at most, a very small effect on sound source localization accuracy. These data suggest that amplitude modulation, while providing information about interaural time differences in headphone studies, does not have much influence on sound source localization accuracy in a sound field.

Spectrotemporal weighting of binaural cues: Effects of a diotic interferer on discrimination of dynamic interaural differences

Spatial judgments are often dominated by low-frequency binaural cues and onset cues when binaural cues vary across the spectrum and duration, respectively, of a brief sound. This study combined these dimensions to assess the spectrotemporal weighting of binaural information. Listeners discriminated target interaural time difference (ITD) and interaural level difference (ILD) carried by the onset, offset, or full duration of a 4-kHz Gabor click train with a 2-ms period in the presence or absence of a diotic 500-Hz interferer tone. ITD and ILD thresholds were significantly elevated by the interferer in all conditions and by a similar amount to previous reports for static cues. Binaural interference was dramatically greater for ITD targets lacking onset cues compared to onset and full-duration conditions. Binaural interference for ILD targets was similar across dynamic-cue conditions. These effects mirror the baseline discriminability of dynamic ITD and ILD cues [Stecker and Brown. (2010). J. Acoust. Soc. Am. 127, 3092-3103], consistent with stronger interference for less-robust/higher-variance cues. The results support the view that binaural cue integration occurs simultaneously across multiple variance-weighted dimensions, including time and frequency.

Models of the electrically stimulated binaural system: A review

In an increasing number of countries, the standard treatment for deaf individuals is moving toward the implantation of two cochlear implants. Today's device technology and fitting procedure, however, appears as if the two implants would serve two independent ears and brains. Many experimental studies have demonstrated that after careful matching and balancing of left and right stimulation in controlled laboratory studies most patients have almost normal sensitivity to interaural level differences and some sensitivity to interaural time differences (ITDs). Mechanisms underlying the limited ITD sensitivity are still poorly understood and many different aspects may contribute. Recent pioneering computational approaches identified some of the functional implications the electric input imposes on the neural brainstem circuits. Simultaneously these studies have raised new questions and certainly demonstrated that further refinement of the model stages is necessary. They join the experimental study's conclusions that binaural device technology, binaural fitting, specific speech coding strategies, and binaural signal processing algorithms are obviously missing components to maximize the benefit of bilateral implantation. Within this review, the existing models of the electrically stimulated binaural system are explained, compared, and discussed from a viewpoint of a "CI device with auditory system" and from that of neurophysiological research.

A method to enhance the use of interaural time differences for cochlear implants in reverberant environments

The ability of normal-hearing (NH) listeners to exploit interaural time difference (ITD) cues conveyed in the modulated envelopes of high-frequency sounds is poor compared to ITD cues transmitted in the temporal fine structure at low frequencies. Sensitivity to envelope ITDs is further degraded when envelopes become less steep, when modulation depth is reduced, and when envelopes become less similar between the ears, common factors when listening in reverberant environments. The vulnerability of envelope ITDs is particularly problematic for cochlear implant (CI) users, as they rely on information conveyed by slowly varying amplitude envelopes. Here, an approach to improve access to envelope ITDs for CIs is described in which, rather than attempting to reduce reverberation, the perceptual saliency of cues relating to the source is increased by selectively sharpening peaks in the amplitude envelope judged to contain reliable ITDs. Performance of the algorithm with room reverberation was assessed through simulating listening with bilateral CIs in headphone experiments with NH listeners. Relative to simulated standard CI processing, stimuli processed with the algorithm generated lower ITD discrimination thresholds and increased extents of laterality. Depending on parameterization, intelligibility was unchanged or somewhat reduced. The algorithm has the potential to improve spatial listening with CIs.

Advancing Binaural Cochlear Implant Technology

This special issue contains a collection of 13 papers highlighting the collaborative research and engineering project entitled Advancing Binaural Cochlear Implant Technology-ABCIT-as well as research spin-offs from the project. In this introductory editorial, a brief history of the project is provided, alongside an overview of the studies.