The role of envelope statistics in detecting changes in interaural correlation

Review of Binaural Processing With Asymmetrical Hearing Outcomes in Patients With Bilateral Cochlear Implants

Bilateral cochlear implants (BiCIs) result in several benefits, including improvements in speech understanding in noise and sound source localization. However, the benefit bilateral implants provide among recipients varies considerably across individuals. Here we consider one of the reasons for this variability: difference in hearing function between the two ears, that is, interaural asymmetry. Thus far, investigations of interaural asymmetry have been highly specialized within various research areas. The goal of this review is to integrate these studies in one place, motivating future research in the area of interaural asymmetry. We first consider bottom-up processing, where binaural cues are represented using excitation-inhibition of signals from the left ear and right ear, varying with the location of the sound in space, and represented by the lateral superior olive in the auditory brainstem. We then consider top-down processing via predictive coding, which assumes that perception stems from expectations based on context and prior sensory experience, represented by cascading series of cortical circuits. An internal, perceptual model is maintained and updated in light of incoming sensory input. Together, we hope that this amalgamation of physiological, behavioral, and modeling studies will help bridge gaps in the field of binaural hearing and promote a clearer understanding of the implications of interaural asymmetry for future research on optimal patient interventions.

Responses to dichotic tone-in-noise stimuli in the inferior colliculus

Human listeners are more sensitive to tones embedded in diotic noise when the tones are out-of-phase at the two ears (N0Sπ) than when they are in-phase (N0S0). The difference between the tone-detection thresholds for these two conditions is referred to as the binaural masking level difference (BMLD) and reflects a benefit of binaural processing. Detection in the N0Sπ condition has been explained in modeling studies by changes in interaural correlation (IAC), but this model has only been directly tested physiologically for low frequencies. Here, the IAC-based hypothesis for binaural detection was examined across a wide range of frequencies and masker levels using recordings in the awake rabbit inferior colliculus (IC). IAC-based cues were strongly correlated with neural responses to N0Sπ stimuli. Additionally, average rate-based thresholds were calculated for both N0S0 and N0Sπ conditions. The rate-based neural BMLD at 500 Hz matched rabbit behavioral data, but the trend of neural BMLDs across frequency differed from that of humans.

The effect of target and interferer frequency on across-frequency binaural interference of interaural-level-difference sensitivity

Across-frequency binaural interference occurs when the sensitivity to changes in interaural differences in a target sound is decreased by a spectrally remote diotic interfering sound. For interaural time differences (ITDs), low-frequency (e.g., 0.5 kHz) interferers cause more interference on high-frequency (e.g., 4 kHz) targets than vice versa. For interaural level differences (ILDs), however, it is unclear if a frequency dependence exists. Therefore, ILD discrimination thresholds and across-frequency binaural interference were measured for target and interferer frequencies between 0.5 and 8 kHz (for tones) or 12 kHz (for narrowband noises). For tones, 8-kHz targets experienced the least interference and 8-kHz interferers produced the most interference, suggesting that higher-frequency ILDs are a more heavily weighted localization cue than lower-frequency ILDs. For narrowband noises, the frequency-dependent interference patterns increased in complexity in comparison to tones. Low-frequency ITD dominance (from randomly varying onset ITDs) and grouping cues (e.g., envelope modulations) might explain some of the complexity in the interference patterns for the noises. These data contribute to a better understanding of across-frequency ILD processing, which remains poorly understood.

Responses to diotic tone-in-noise stimuli in the inferior colliculus: stimulus envelope and neural fluctuation cues

Human detection thresholds in tone-in-noise (TIN) paradigms cannot be explained by the prevalent power-spectrum model when stimulus energy is made less reliable, e.g., in roving-level or equal-energy paradigms. Envelope-related cues provide an alternative that is more robust across level. The TIN stimulus envelope is encoded by slow fluctuations in auditory-nerve (AN) responses - a temporal representation affected by inner-hair-cell (IHC) saturation and cochlear compression. Here, envelope-related fluctuations in AN responses were hypothesized to be reflected in responses of neurons in the inferior colliculus (IC), which have average discharge rates that are sensitive to amplitude-modulation (AM) depth and frequency. Responses to tones masked by narrowband gaussian noise (GN) and low-noise noise (LNN) were recorded in the IC of awake rabbits. Fluctuation amplitudes in the stimulus envelope and in model AN responses decrease for GN maskers and increase for LNN upon addition of tones near threshold. Response rates of IC neurons that are excited by AM were expected to be positively correlated with fluctuation amplitudes, whereas rates of neurons suppressed by AM were expected to be negatively correlated. Of neurons with measurable TIN-detection thresholds, most had the predicted changes in rate with increasing tone level for both GN and LNN maskers. Changes in rate with tone level were correlated with envelope sensitivity measured with two methods, including the maximum slopes of modulation transfer functions. IC rate-based thresholds were broadly consistent with published human and rabbit behavioral data. These results highlight the importance of midbrain sensitivity to envelope cues, as represented in peripheral neural fluctuations, for detection of signals in noise.

Impaired interaural correlation processing in people with schizophrenia

Detection of transient changes in interaural correlation is based on the temporal precision of the central representations of acoustic signals. Whether schizophrenia impairs the temporal precision in the interaural correlation process is not clear. In both participants with schizophrenia and matched healthy-control participants, this study examined the detection of a break in interaural correlation (BIC, a change in interaural correlation from 1 to 0 and back to 1), including the longest interaural delay at which a BIC was just audible, representing the temporal extent of the primitive auditory memory (PAM). Moreover, BIC-induced electroencephalograms (EEGs) and the relationships between the early binaural psychoacoustic processing and higher cognitive functions, which were assessed by the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), were examined. The results showed that compared to healthy controls, participants with schizophrenia exhibited poorer BIC detection, PAM and RBANS score. Both the BIC-detection accuracy and the PAM extent were correlated with the RBANS score. Moreover, participants with schizophrenia showed weaker BIC-induced N1-P2 amplitude which was correlated with both theta-band power and inter-trial phase coherence. These results suggested that schizophrenia impairs the temporal precision of the central representations of acoustic signals, affecting both interaural correlation processing and higher-order cognitions.

The effect of envelope modulations on binaural processing

An experiment was performed with 10 young normal-hearing listeners that attempted to determine if envelope modulations affected binaural processing in bandlimited pulse trains. Listeners detected an interaurally out-of-phase carrier pulse train in the presence of different amplitude modulations. The peaks of the pulses were constant (called "flat" or F), followed envelope modulations from an interaurally correlated 50-Hz bandwidth noise (called CM), or followed modulations from an interaurally uncorrelated noise (called UM). The pulse rate was varied from 50 to 500 pulses per second (pps) and the center frequency (CF) was 4 or 8 kHz. It was hypothesized that CM would cause no change or an increase in performance compared to F; UM would cause a decrease because of the blurring of the binaural detection cue. There was a small but significant decrease from F to CM (inconsistent with the hypothesis) and a further decrease from CM to UM (consistent with the hypothesis). Critically, there was a significant envelope by rate interaction caused by a decrease from F to CM for the 200-300 pps rates. The data can be explained by a subject-based factor, where some listeners experienced interaural envelope decorrelation when the sound was encoded by the auditory system that reduced performance when the modulations were present. Since the decrease in performance between F and CM conditions was small, it seems that most young normal-hearing listeners have very similar encoding of modulated stimuli across the ears. This type of task, when further optimized, may be able to assess if hearing-impaired populations experience interaural decorrelation from encoding modulated stimuli and therefore could help better understand the limited spatial hearing in populations like cochlear-implant users.

Binaural unmasking with temporal envelope and fine structure in listeners with cochlear implants

For normal-hearing (NH) listeners, interaural information in both temporal envelope and temporal fine structure contribute to binaural unmasking of target signals in background noise; however, in many conditions low-frequency interaural information in temporal fine structure produces greater binaural unmasking. For bilateral cochlear-implant (CI) listeners, interaural information in temporal envelope contributes to binaural unmasking; however, the effect of encoding temporal fine structure information in electrical pulse timing (PT) is not fully understood. In this study, diotic and dichotic signal detection thresholds were measured in CI listeners using bilaterally synchronized single-electrode stimulation for conditions in which the temporal envelope was presented without temporal fine structure encoded (constant-rate pulses) or with temporal fine structure encoded (pulses timed to peaks of the temporal fine structure). CI listeners showed greater binaural unmasking at 125 pps with temporal fine structure encoded than without. There was no significant effect of encoding temporal fine structure at 250 pps. A similar pattern of performance was shown by NH listeners presented with acoustic pulse trains designed to simulate CI stimulation. The results suggest a trade-off across low rates between interaural information obtained from temporal envelope and that obtained from temporal fine structure encoded in PT.

Cortical Correlates of Binaural Temporal Processing Deficits in Older Adults

OBJECTIVES

This study was designed to evaluate binaural temporal processing in young and older adults using a binaural masking level difference (BMLD) paradigm. Using behavioral and electrophysiological measures within the same listeners, a series of stimulus manipulations was used to evaluate the relative contribution of binaural temporal fine-structure and temporal envelope cues. We evaluated the hypotheses that age-related declines in the BMLD task would be more strongly associated with temporal fine-structure than envelope cues and that age-related declines in behavioral measures would be correlated with cortical auditory evoked potential (CAEP) measures.

DESIGN

Thirty adults participated in the study, including 10 young normal-hearing, 10 older normal-hearing, and 10 older hearing-impaired adults with bilaterally symmetric, mild-to-moderate sensorineural hearing loss. Behavioral and CAEP thresholds were measured for diotic (So) and dichotic (Sπ) tonal signals presented in continuous diotic (No) narrowband noise (50-Hz wide) maskers. Temporal envelope cues were manipulated by using two different narrowband maskers; Gaussian noise (GN) with robust envelope fluctuations and low-noise noise (LNN) with minimal envelope fluctuations. The potential to use temporal fine-structure cues was controlled by varying the signal frequency (500 or 4000 Hz), thereby relying on the natural decline in phase-locking with increasing frequency.

RESULTS

Behavioral and CAEP thresholds were similar across groups for diotic conditions, while the masking release in dichotic conditions was larger for younger than for older participants. Across all participants, BMLDs were larger for GN than LNN and for 500-Hz than for 4000-Hz conditions, where envelope and fine-structure cues were most salient, respectively. Specific age-related differences were demonstrated for 500-Hz dichotic conditions in GN and LNN, reflecting reduced binaural temporal fine-structure coding. No significant age effects were observed for 4000-Hz dichotic conditions, consistent with similar use of binaural temporal envelope cues across age in these conditions. For all groups, thresholds and derived BMLD values obtained using the behavioral and CAEP methods were strongly correlated, supporting the notion that CAEP measures may be useful as an objective index of age-related changes in binaural temporal processing.

CONCLUSIONS

These results demonstrate an age-related decline in the processing of binaural temporal fine-structure cues with preserved temporal envelope coding that was similar with and without mild-to-moderate peripheral hearing loss. Such age-related changes can be reliably indexed by both behavioral and CAEP measures in young and older adults.

Temporal dynamics and uncertainty in binaural hearing revealed by anticipatory eye movements

Accurate perception of binaural cues is essential for left-right sound localization. Much literature focuses on threshold measures of perceptual acuity and accuracy. This study focused on supra-threshold perception using an anticipatory eye movement (AEM) paradigm designed to capture subtle aspects of perception that might not emerge in behavioral-motor responses, such as the accumulation of certainty, and rapid revisions in decision-making. Participants heard interaural timing differences (ITDs) or interaural level differences in correlated or uncorrelated narrowband noises, respectively. A cartoon ball moved behind an occluder and then emerged from the left or right side, consistent with the binaural cue. Participants anticipated the correct answer (before it appeared) by looking where the ball would emerge. Results showed quicker and more steadfast gaze fixations for stimuli with larger cue magnitudes. More difficult stimuli elicited a wider distribution of saccade times and greater number of corrective saccades before final judgment, implying perceptual uncertainty or competition. Cue levels above threshold elicited some wrong-way saccades that were quickly corrected. Saccades to ITDs were earlier and more reliable for low-frequency noises. The AEM paradigm reveals the time course of uncertainty and changes in perceptual decision-making for supra-threshold binaural stimuli even when behavioral responses are consistently correct.

Age-related differences in binaural masking level differences: behavioral and electrophysiological evidence

The effects of aging and stimulus configuration on binaural masking level differences (BMLDs) were measured behaviorally and electrophysiologically, using the frequency-following response (FFR) to target brainstem/midbrain encoding. The tests were performed in 15 younger normal-hearing (<30 yr) and 15 older normal-hearing (>60 yr) participants. The stimuli consisted of a 500-Hz target tone embedded in a narrowband (50-Hz bandwidth) or wideband (1,500-Hz bandwidth) noise masker. The interaural phase conditions included NoSo (tone and noise presented interaurally in-phase), NoSπ (noise presented interaurally in-phase and tone presented out-of-phase), and NπSo (noise presented interaurally out-of-phase and tone presented in-phase) configurations. In the behavioral experiment, aging reduced the magnitude of the BMLD. The magnitude of the BMLD was smaller for the NoSo-NπSo threshold difference compared with the NoSo-NoSπ threshold difference, and it was also smaller in narrowband compared with wideband conditions, consistent with previous measurements. In the electrophysiology experiment, older participants had reduced FFR magnitudes and smaller differences between configurations. There were significant changes in FFR magnitude between the NoSo to NoSπ configurations but not between the NoSo to NπSo configurations. The age-related reduction in FFR magnitudes suggests a temporal processing deficit, but no correlation was found between FFR magnitudes and behavioral BMLDs. Therefore, independent mechanisms may be contributing to the behavioral and neural deficits. Specifically, older participants had higher behavioral thresholds than younger participants for the NoSπ and NπSo configurations but had equivalent thresholds for the NoSo configuration. However, FFR magnitudes were reduced in older participants across all configurations. NEW & NOTEWORTHY Behavioral and electrophysiological testing reveal an aging effect for stimuli presented in wideband and narrowband noise conditions, such that behavioral binaural masking level differences and subcortical spectral magnitudes are reduced in older compared with younger participants. These deficits in binaural processing may limit the older participant's ability to use spatial cues to understand speech in environments containing competing sound sources.

Binaural release from masking with single- and multi-electrode stimulation in children with cochlear implants

Cochlear implants (CIs) provide children with access to speech information from a young age. Despite bilateral cochlear implantation becoming common, use of spatial cues in free field is smaller than in normal-hearing children. Clinically fit CIs are not synchronized across the ears; thus binaural experiments must utilize research processors that can control binaural cues with precision. Research to date has used single pairs of electrodes, which is insufficient for representing speech. Little is known about how children with bilateral CIs process binaural information with multi-electrode stimulation. Toward the goal of improving binaural unmasking of speech, this study evaluated binaural unmasking with multi- and single-electrode stimulation. Results showed that performance with multi-electrode stimulation was similar to the best performance with single-electrode stimulation. This was similar to the pattern of performance shown by normal-hearing adults when presented an acoustic CI simulation. Diotic and dichotic signal detection thresholds of the children with CIs were similar to those of normal-hearing children listening to a CI simulation. The magnitude of binaural unmasking was not related to whether the children with CIs had good interaural time difference sensitivity. Results support the potential for benefits from binaural hearing and speech unmasking in children with bilateral CIs.

Untrained listeners experience difficulty detecting interaural correlation changes in narrowband noises

Interaural correlation change detection was measured in untrained normal-hearing listeners. Narrowband (10-Hz) noises were varied by center frequency (CF; 500 or 4000 Hz) and diotic level roving (absent or present). For the 500-Hz CF, 96% of listeners could achieve threshold (79.4% correct at the easiest testing level) if roving was absent, but only 36% of listeners could if level roving was present. No one could achieve threshold at the 4000-Hz CF, unlike trained listeners in the literature. The results raise questions about how individual differences affect learning and generalization of monaural and binaural cues related to interaural correlation detection.

Interaural envelope correlation change discrimination in bilateral cochlear implantees: effects of mismatch, centering, and onset of deafness

Bilateral cochlear implant (CI) listeners can perform binaural tasks, but they are typically worse than normal-hearing (NH) listeners. To understand why this difference occurs and the mechanisms involved in processing dynamic binaural differences, interaural envelope correlation change discrimination sensitivity was measured in real and simulated CI users. In experiment 1, 11 CI (eight late deafened, three early deafened) and eight NH listeners were tested in an envelope correlation change discrimination task. Just noticeable differences (JNDs) were best for a matched place-of-stimulation and increased for an increasing mismatch. In experiment 2, attempts at intracranially centering stimuli did not produce lower JNDs. In experiment 3, the percentage of correct identifications of antiphasic carrier pulse trains modulated by correlated envelopes was measured as a function of mismatch and pulse rate. Sensitivity decreased for increasing mismatch and increasing pulse rate. The experiments led to two conclusions. First, envelope correlation change discrimination necessitates place-of-stimulation matched inputs. However, it is unclear if previous experience with acoustic hearing is necessary for envelope correlation change discrimination. Second, NH listeners presented with CI simulations demonstrated better performance than real CI listeners. If the simulations are realistic representations of electrical stimuli, real CI listeners appear to have difficulty processing interaural information in modulated signals.

Sensitivity to interaural envelope correlation changes in bilateral cochlear-implant users

Provision of bilateral cochlear implants (CIs) to people who are deaf is partially justified by improved abilities to understand speech in noise when comparing bilateral vs unilateral listening conditions. However, bilateral CI listeners generally show only monaural head shadow with little improvement in speech understanding due to binaural unmasking. Sensitivity to change in interaural envelope correlation, which is related to binaural speech unmasking, was investigated. Bilateral CI users were tested with bilaterally synchronized processors at single, pitch-matched electrode pairs. First, binaural masking level differences (BMLDs) were measured using 1000 pulse-per-second (pps) carriers, yielding BMLDs of 11.1 ± 6.5 and 8.5 ± 4.2 dB for 10- and 50-Hz bandwidth masking noises, respectively. Second, envelope correlation change just-noticeable differences (JNDs) were measured. Stimuli presented at 1000 pps yielded lower JNDs than those presented at 100 pps. Furthermore, perfectly correlated reference stimuli produced lower JNDs than uncorrelated references, and uncorrelated references generally produced immeasurable JNDs. About 25% of JNDs measured in the CI listeners were in the range of JNDs observed in normal-hearing listeners presented CI simulations. In conclusion, CI listeners can perceive changes in interaural envelope correlation, but the poor performance may be a major limiting factor in binaural unmasking tested to date in realistic listening environments.

Accounting for binaural detection as a function of masker interaural correlation: effects of center frequency and bandwidth

Binaural detection was measured as a function of the center frequency, bandwidth, and interaural correlation of masking noise. Thresholds were obtained for 500-Hz or 125-Hz Sπ tonal signals and for the latter stimuli (noise or signal-plus-noise) transposed to 4 kHz. A primary goal was assessment of the generality of van der Heijden and Trahiotis' [J. Acoust. Soc. Am. 101, 1019-1022 (1997)] hypothesis that thresholds could be accounted for by the "additive" masking effects of the underlying No and Nπ components of a masker having an interaural correlation of ρ. Results indicated that (1) the overall patterning of the data depended neither upon center frequency nor whether information was conveyed via the waveform or by its envelope; (2) thresholds for transposed stimuli improved relative to their low-frequency counterparts as bandwidth of the masker was increased; (3) the additivity approach accounted well for the data across stimulus conditions but consistently overestimated MLDs, especially for narrowband maskers; (4) a quantitative approach explicitly taking into account the distributions of time-varying ITD-based lateral positions produced by masker-alone and signal-plus-masker waveforms proved more successful, albeit while employing a larger set of assumptions, parameters, and computational complexity.

The effect of interaural fluctuation rate on correlation change discrimination

While bilateral cochlear implants (CIs) provide some binaural benefits, these benefits are limited compared to those observed in normal-hearing (NH) listeners. The large frequency-to-electrode allocation bandwidths (BWs) in CIs compared to auditory filter BWs in NH listeners increases the interaural fluctuation rate available for binaural unmasking, which may limit binaural benefits. The purpose of this work was to investigate the effect of interaural fluctuation rate on correlation change discrimination and binaural masking-level differences in NH listeners presented a CI simulation using a pulsed-sine vocoder. In experiment 1, correlation-change just-noticeable differences (JNDs) and tone-in-noise thresholds were measured for narrowband noises with different BWs and center frequencies (CFs). The results suggest that the BW, CF, and/or interaural fluctuation rate are important factors for correlation change discrimination. In experiment 2, the interaural fluctuation rate was systematically varied and dissociated from changes in BW and CF by using a pulsed-sine vocoder. Results indicated that the interaural fluctuation rate did not affect correlation change JNDs for correlated reference noises; however, slow interaural fluctuations increased correlation change JNDs for uncorrelated reference noises. In experiment 3, the BW, CF, and vocoder pulse rate were varied while interaural fluctuation rate was held constant. JNDs increased for increasing BW and decreased for increasing CF. In summary, relatively fast interaural fluctuation rates are not detrimental for detecting changes in interaural correlation. Thus, limiting factors to binaural benefits in CI listeners could be a result of other temporal and/or spectral deficiencies from electrical stimulation.

Binaural detection with narrowband and wideband reproducible noise maskers. IV. Models using interaural time, level, and envelope differences

The addition of out-of-phase tones to in-phase noises results in dynamic interaural level difference (ILD) and interaural time difference (ITD) cues for the dichotic tone-in-noise detection task. Several models have been used to predict listeners' detection performance based on ILD, ITD, or different combinations of the two cues. The models can be tested using detection performance from an ensemble of reproducible-noise maskers. Previous models cannot predict listeners' detection performance for reproducible-noise maskers without fitting the data. Here, two models were tested for narrowband and wideband reproducible-noise experiments. One model was a linear combination of ILD and ITD that included the generally ignored correlation between the two cues. The other model was based on a newly proposed cue, the slope of the interaural envelope difference (SIED). Predictions from both models explained a significant portion of listeners' performance for detection of a 500-Hz tone in wideband noise. Predictions based on the SIED approached the predictable variance in the wideband condition. The SIED represented a nonlinear combination of ILD and ITD, with the latter cue dominating. Listeners did not use a common strategy (cue) to detect tones in the narrowband condition and may use different single frequencies or different combinations of frequency channels.

Comparison of across-frequency integration strategies in a binaural detection model

Breebaart et al. [J. Acoust. Soc. Am. 110, 1089-1104 (2001)] reported that the masker bandwidth dependence of detection thresholds for an out-of-phase signal and an in-phase noise masker (N0Sπ) can be explained by principles of integration of information across critical bands. In this paper, different methods for such across-frequency integration process are evaluated as a function of the bandwidth and notch width of the masker. The results indicate that an "optimal detector" model assuming independent internal noise in each critical band provides a better fit to experimental data than a best filter or a simple across-frequency integrator model. Furthermore, the exponent used to model peripheral compression influences the accuracy of predictions in notched conditions.

Effect of mismatched place-of-stimulation on the salience of binaural cues in conditions that simulate bilateral cochlear-implant listening

Although bilateral cochlear implantation has the potential to improve sound localization and speech understanding in noise, obstacles exist in presenting maximally useful binaural information to bilateral cochlear-implant (CI) users. One obstacle is that electrode arrays may differ in cochlear position by several millimeters, thereby stimulating different neural populations. Effects of interaural frequency mismatch on binaural processing were studied in normal-hearing (NH) listeners using band-limited pulse trains, thereby avoiding confounding factors that may occur in CI users. In experiment 1, binaural image fusion was measured to capture perceptual number, location, and compactness. Subjects heard a single, compact image on 73% of the trials. In experiment 2, intracranial image location was measured for different interaural time differences (ITDs) and interaural level differences (ILDs). For larger mismatch, locations perceptually shifted towards the ear with the higher carrier frequency. In experiment 3, ITD and ILD just-noticeable differences (JNDs) were measured. JNDs increased with decreasing bandwidth and increasing mismatch, but were always measurable up to 3 mm of mismatch. If binaural-hearing mechanisms are similar between NH and CI subjects, these results may explain reduced sensitivity of ITDs and ILDs in CI users. Large mismatches may lead to distorted spatial maps and reduced binaural image fusion.