Lateralization of stimuli with independent fine-structure and envelope-based temporal disparities

A unitary model of auditory frequency change perception

Changes in the frequency content of sounds over time are arguably the most basic form of information about the behavior of sound-emitting objects. In perceptual studies, such changes have mostly been investigated separately, as aspects of either pitch or timbre. Here, we propose a unitary account of "up" and "down" subjective judgments of frequency change, based on a model combining auditory correlates of acoustic cues in a sound-specific and listener-specific manner. To do so, we introduce a generalized version of so-called Shepard tones, allowing symmetric manipulations of spectral information on a fine scale, usually associated to pitch (spectral fine structure, SFS), and on a coarse scale, usually associated timbre (spectral envelope, SE). In a series of behavioral experiments, listeners reported "up" or "down" shifts across pairs of generalized Shepard tones that differed in SFS, in SE, or in both. We observed the classic properties of Shepard tones for either SFS or SE shifts: subjective judgements followed the smallest log-frequency change direction, with cases of ambiguity and circularity. Interestingly, when both SFS and SE changes were applied concurrently (synergistically or antagonistically), we observed a trade-off between cues. Listeners were encouraged to report when they perceived "both" directions of change concurrently, but this rarely happened, suggesting a unitary percept. A computational model could accurately fit the behavioral data by combining different cues reflecting frequency changes after auditory filtering. The model revealed that cue weighting depended on the nature of the sound. When presented with harmonic sounds, listeners put more weight on SFS-related cues, whereas inharmonic sounds led to more weight on SE-related cues. Moreover, these stimulus-based factors were modulated by inter-individual differences, revealing variability across listeners in the detailed recipe for "up" and "down" judgments. We argue that frequency changes are tracked perceptually via the adaptive combination of a diverse set of cues, in a manner that is in fact similar to the derivation of other basic auditory dimensions such as spatial location.

Frequency dependence of sensitivity to interaural phase differences in pure tones

It is well established that in normal-hearing humans, the threshold of interaural time differences for pure tones increases dramatically above about 1300 Hz, only to become unmeasurable above 1400 Hz. However, physiological data and auditory models suggest that the actual decline in sensitivity is more gradual and only appears to be abrupt because the maximum of the psychometric function dips below the threshold proportion correct, e.g., 0.794. Published data only report thresholds at certain proportions correct but not the decline of proportions correct or of the sensitivity index d' with increasing frequencies. Here, we present pure-tone behavioral data obtained with a constant stimulus procedure. Seven of nine subjects showed proportions correct above 0.9 at 1300 Hz and virtually no sensitivity at 1500 Hz (proportion correct within 0.07 of chance level). This corresponds to a sensitivity decline of 46-78 dB/oct, much steeper than predicted by existing models or by the decline of phase locking of the auditory nerve fibers in animal data.

Sensitivity to Envelope Interaural Time Differences: Modeling Auditory Modulation Filtering

For amplitude-modulated sound, the envelope interaural time difference (ITD_ENV) is a potential cue for sound-source location. ITD_ENV is encoded in the lateral superior olive (LSO) of the auditory brainstem, by excitatory-inhibitory (EI) neurons receiving ipsilateral excitation and contralateral inhibition. Between human listeners, sensitivity to ITD_ENV varies considerably, but ultimately decreases with increasing stimulus carrier frequency, and decreases more strongly with increasing modulation rate. Mechanisms underlying the variation in behavioral sensitivity remain unclear. Here, with increasing carrier frequency (4-10 kHz), as we phenomenologically model the associated decrease in ITD_ENV sensitivity using arbitrarily fewer neurons consistent across populations, we computationally model the variable sensitivity across human listeners and modulation rates (32-800 Hz) as the decreasing range of membrane frequency responses in LSO neurons. Transposed tones stimulate a bilateral auditory-periphery model, driving model EI neurons where electrical membrane impedance filters the frequency content of inputs driven by amplitude-modulated sound, evoking modulation filtering. Calculated from Fisher information in spike-rate functions of ITD_ENV, for model EI neuronal populations distinctly reflecting the LSO range in membrane frequency responses, just-noticeable differences in ITD_ENV collectively reproduce the largest variation in ITD_ENV sensitivity across human listeners. These slow to fast model populations each generally match the best human ITD_ENV sensitivity at a progressively higher modulation rate, by membrane-filtering and spike-generation properties producing realistically less than Poisson variance. Non-resonant model EI neurons are also sensitive to interaural intensity differences. With peripheral filters centered between carrier frequency and modulation sideband, fast resonant model EI neurons extend ITD_ENV sensitivity above 500-Hz modulation.

Influence of envelope fluctuation on the lateralization of interaurally delayed low-frequency stimuli

Disregarding onset and offset effects, interaurally delaying a 500 Hz tone by 1.5 ms is identical to advancing it by 0.5 ms. When presented over headphones, humans indeed perceive such a tone lateralized toward the side of the nominal lag. Any stimulus other than a tone has more than one frequency component and is thus unambiguous. It has been shown that phase ambiguity can be resolved when increasing the stimulus bandwidth. This has mostly been attributed to the integration of information across frequencies. Additionally, interaural timing information conveyed in the stimulus envelope within a single frequency channel is a second possible cue that could help to resolve phase ambiguity. This study employs stimuli designed to differ in the amount of envelope fluctuation while retaining the same power spectral density as well as interaural differences. Any difference in lateralization must thus be a result of the difference in envelope. The results show that stimuli with strong envelope fluctuation require significantly smaller bandwidths to resolve phase ambiguity when compared to stimuli with weak envelope fluctuation. This suggests that within-channel information is an important cue used to resolve phase ambiguity.

Reducing the Device Delay Mismatch Can Improve Sound Localization in Bimodal Cochlear Implant/Hearing-Aid Users

In users of a cochlear implant (CI) together with a contralateral hearing aid (HA), so-called bimodal listeners, differences in processing latencies between digital HA and CI up to 9 ms constantly superimpose interaural time differences. In the present study, the effect of this device delay mismatch on sound localization accuracy was investigated. For this purpose, localization accuracy in the frontal horizontal plane was measured with the original and minimized device delay mismatch. The reduction was achieved by delaying the CI stimulation according to the delay of the individually worn HA. For this, a portable, programmable, battery-powered delay line based on a ring buffer running on a microcontroller was designed and assembled. After an acclimatization period to the delayed CI stimulation of 1 hr, the nine bimodal study participants showed a highly significant improvement in localization accuracy of 11.6% compared with the everyday situation without the delay line ( p < .01). Concluding, delaying CI stimulation to minimize the device delay mismatch seems to be a promising method to increase sound localization accuracy in bimodal listeners.

The fMRI Data of Thompson et al. (2006) Do Not Constrain How the Human Midbrain Represents Interaural Time Delay

This commentary provides an alternate interpretation of the fMRI data that were presented in a communication to the journal Nature Neuroscience (Thompson et al., Nat. Neurosci. 9: 1096-1098, 2006 ). The authors argued that their observations demonstrated that traditional models of binaural hearing which incorporate "internal delays," such as the coincidence-counting mechanism proposed by Jeffress and quantified by Colburn, are invalid, and that a new model for human interaural time delay processing must be developed. We argue that the fMRI data presented do not strongly favor either the refutation or the retention of the traditional models, although they may be useful in constraining the physiological sites of various processing stages. The conclusions of Thompson et al. are based on the locations of maximal activity in the midbrain in response to selected binaural signals. These locations are inconsistent with well-known perceptual attributes of the stimuli under consideration, as is noted by the authors, which suggests that further processing is involved in forming the percept of subjective lateral position.

Functional rate-code models of the auditory brainstem for predicting lateralization and discrimination data of human binaural perception

This paper presents a rate-code model of binaural interaction inspired by recent neurophysiological findings. The model consists of a peripheral part and a binaural part. The binaural part is composed of models of the medial superior olive (MSO) and the lateral superior olive (LSO), which are parts of the auditory brainstem. The MSO and LSO model outputs are preprocessed in the interaural time difference (ITD) and interaural level difference (ILD) central stages, respectively, which give absolute values of the predicted lateralization at their outputs, allowing a direct comparison with psychophysical data. The predictions obtained with the MSO and LSO models are compared with subjective data on the lateralization of pure tones and narrowband noises, discrimination of the ITD and ILD, and discrimination of the phase warp. The lateralization and discrimination experiments show good agreement with the subjective data. In the case of the phase-warp experiment, the models agree qualitatively with the subjective data. The results demonstrate that rate-code models of MSO and LSO can be used to explain psychophysical data considering lateralization and discrimination based on binaural cues.

A framework for computational modelling of interaural time difference discrimination of normal and hearing-impaired listeners

Different computational models have been developed to study the interaural time difference (ITD) perception. However, only few have used a physiologically inspired architecture to study ITD discrimination. Furthermore, they do not include aspects of hearing impairment. In this work, a framework was developed to predict ITD thresholds in listeners with normal and impaired hearing. It combines the physiologically inspired model of the auditory periphery proposed by Zilany, Bruce, Nelson, and Carney [(2009). J. Acoust. Soc. Am. 126(5), 2390-2412] as a front end with a coincidence detection stage and a neurometric decision device as a back end. It was validated by comparing its predictions against behavioral data for narrowband stimuli from literature. The framework is able to model ITD discrimination of normal-hearing and hearing-impaired listeners at a group level. Additionally, it was used to explore the effect of different proportions of outer- and inner-hair cell impairment on ITD discrimination.

Processing of interaural phase differences in components of harmonic and mistuned complexes in the inferior colliculus of the Mongolian gerbil

Harmonicity and spatial location provide eminent cues for the perceptual grouping of sounds. In general, harmonicity is a strong grouping cue. In contrast, spatial cues such as interaural phase or time difference provide for strong grouping of stimulus sequences but weak grouping for simultaneously presented sounds. By studying the neuronal basis underlying the interaction of these cues in processing simultaneous sounds using van Rossum spike train distance measures, we aim at explaining the interaction observed in psychophysical experiments. Responses to interaural phase differences imposed on single components of harmonic and mistuned complex tones as well as noise delay functions were recorded as multiunit responses from the inferior colliculus of Mongolian gerbils. Results revealed a better representation of interaural phase differences if imposed on a harmonic rather than a mistuned frequency component of a complex tone. The representation of interaural phase differences was better for long integration-time windows approximately reflecting firing rates rather than short integration-time windows reflecting the temporal pattern of the stimulus-driven response. We found only a weak impact of interaural phase differences if combined with mistuning of a component in a harmonic tone complex.

Effects of age on sensitivity to interaural time differences in envelope and fine structure, individually and in combination

Sensitivity to interaural time differences (ITDs) in envelope and temporal fine structure (TFS) of amplitude-modulated (AM) tones was assessed for young and older subjects, all with clinically normal hearing at the carrier frequencies of 250 and 500 Hz. Some subjects had hearing loss at higher frequencies. In experiment 1, thresholds for detecting changes in ITD were measured when the ITD was present in the TFS alone (ITD_TFS), the envelope alone (ITD_ENV), or both (ITD_TFS/ENV). Thresholds tended to be higher for the older than for the young subjects. ITD_ENV thresholds were much higher than ITD_TFS thresholds, while ITD_TFS/ENV thresholds were similar to ITD_TFS thresholds. ITD_TFS thresholds were lower than ITD thresholds obtained with an unmodulated pure tone, indicating that uninformative AM can improve ITD_TFS discrimination. In experiment 2, equally detectable values of ITD_TFS and ITD_ENV were combined so as to give consistent or inconsistent lateralization. There were large individual differences, but several subjects gave scores that were much higher than would be expected from the optimal combination of independent sources of information, even for the inconsistent condition. It is suggested that ITD_TFS and ITD_ENV cues are processed partly independently, but that both cues influence lateralization judgments, even when one cue is uninformative.

Interaction of interaural cues and their contribution to the lateralisation of Mongolian gerbils (Meriones unguiculatus)

The main sound localisation cues in the horizontal plane are interaural time and level differences (ITDs and ILDs, respectively). ITDs are thought to be the dominant cue in the low-frequency range, ILDs the dominant cue in the high-frequency range. ITDs and ILDs co-occur. Their interaction and contribution to the lateralisation of pure tones by Mongolian gerbils was investigated behaviourally using cross-talk cancellation techniques for presenting ITDs and ILDs independently. First, ITDs were applied to pure tones with frequencies ≤ 2 kHz to the ongoing waveform, at the onsets and offsets, or in both the ongoing waveform and at the onsets and offsets. Gerbils could lateralise tones only if ongoing ITDs were present indicating that ongoing ITDs are decisive for the lateralisation of low-frequency tones. Second, an ITD was added to 2-to-6-kHz tones with varying ILD. Gerbils' lateralisation was unaffected by the ITD indicating that a large ILD provides a strong lateralisation cue at those frequencies. Finally, small ILDs were applied to 2-kHz tones with an ongoing ITD, pointing either to the same or opposing sides as the ITD. Gerbils' lateralisation was driven by the ITD but strongly affected by the ILD indicating that both interaural cues contribute to the lateralisation.

Combination of binaural and harmonic masking release effects in the detection of a single component in complex tones

Both harmonic and binaural signal properties are relevant for auditory processing. To investigate how these cues combine in the auditory system, detection thresholds for an 800-Hz tone masked by a diotic (i.e., identical between the ears) harmonic complex tone were measured in six normal-hearing subjects. The target tone was presented either diotically or with an interaural phase difference (IPD) of 180° and in either harmonic or "mistuned" relationship to the diotic masker. Three different maskers were used, a resolved and an unresolved complex tone (fundamental frequency: 160 and 40 Hz) with four components below and above the target frequency and a broadband unresolved complex tone with 12 additional components. The target IPD provided release from masking in most masker conditions, whereas mistuning led to a significant release from masking only in the diotic conditions with the resolved and the narrowband unresolved maskers. A significant effect of mistuning was neither found in the diotic condition with the wideband unresolved masker nor in any of the dichotic conditions. An auditory model with a single analysis frequency band and different binaural processing schemes was employed to predict the data of the unresolved masker conditions. Sensitivity to modulation cues was achieved by including an auditory-motivated modulation filter in the processing pathway. The predictions of the diotic data were in line with the experimental results and literature data in the narrowband condition, but not in the broadband condition, suggesting that across-frequency processing is involved in processing modulation information. The experimental and model results in the dichotic conditions show that the binaural processor cannot exploit modulation information in binaurally unmasked conditions.

A framework for testing and comparing binaural models

Auditory research has a rich history of combining experimental evidence with computational simulations of auditory processing in order to deepen our theoretical understanding of how sound is processed in the ears and in the brain. Despite significant progress in the amount of detail and breadth covered by auditory models, for many components of the auditory pathway there are still different model approaches that are often not equivalent but rather in conflict with each other. Similarly, some experimental studies yield conflicting results which has led to controversies. This can be best resolved by a systematic comparison of multiple experimental data sets and model approaches. Binaural processing is a prominent example of how the development of quantitative theories can advance our understanding of the phenomena, but there remain several unresolved questions for which competing model approaches exist. This article discusses a number of current unresolved or disputed issues in binaural modelling, as well as some of the significant challenges in comparing binaural models with each other and with the experimental data. We introduce an auditory model framework, which we believe can become a useful infrastructure for resolving some of the current controversies. It operates models over the same paradigms that are used experimentally. The core of the proposed framework is an interface that connects three components irrespective of their underlying programming language: The experiment software, an auditory pathway model, and task-dependent decision stages called artificial observers that provide the same output format as the test subject.

Envelope contributions to the representation of interaural time difference in the forebrain of barn owls

Birds and mammals use the interaural time difference (ITD) for azimuthal sound localization. While barn owls can use the ITD of the stimulus carrier frequency over nearly their entire hearing range, mammals have to utilize the ITD of the stimulus envelope to extend the upper frequency limit of ITD-based sound localization. ITD is computed and processed in a dedicated neural circuit that consists of two pathways. In the barn owl, ITD representation is more complex in the forebrain than in the midbrain pathway because of the combination of two inputs that represent different ITDs. We speculated that one of the two inputs includes an envelope contribution. To estimate the envelope contribution, we recorded ITD response functions for correlated and anticorrelated noise stimuli in the barn owl's auditory arcopallium. Our findings indicate that barn owls, like mammals, represent both carrier and envelope ITDs of overlapping frequency ranges, supporting the hypothesis that carrier and envelope ITD-based localization are complementary beyond a mere extension of the upper frequency limit.NEW & NOTEWORTHY The results presented in this study show for the first time that the barn owl is able to extract and represent the interaural time difference (ITD) information conveyed by the envelope of a broadband acoustic signal. Like mammals, the barn owl extracts the ITD of the envelope and the carrier of a signal from the same frequency range. These results are of general interest, since they reinforce a trend found in neural signal processing across different species.

Extent of lateralization at large interaural time differences in simulated electric hearing and bilateral cochlear implant users

Normal-hearing (NH) listeners are able to localize sound sources with extraordinary accuracy through interaural cues, most importantly interaural time differences (ITDs) in the temporal fine structure. Bilateral cochlear implant (CI) users are also able to localize sound sources, yet generally at lower accuracy than NH listeners. The gap in performance can in part be attributed to current CI systems not faithfully transmitting interaural cues, especially ITDs. With the introduction of binaurally linked CI systems, the presentation of ITD cues for bilateral CI users is foreseeable. The current study therefore investigated extent-of-lateralization percepts elicited in bilateral CI listeners when presented with single-electrode pulse-trains carrying controlled ITD cues. The results were compared against NH listeners listening to broadband stimuli as well as simulations of CI listening. Broadband stimuli in NH listeners were perceived as fully lateralized within the natural ITD range. Using simulated as well as real CI stimuli, however, only a fraction of the full extent of lateralization range was covered by natural ITDs. The maximum extent of lateralization was reached at ITDs as large as twice the natural limit. The results suggest that ITD-enhancement might be a viable option for improving localization abilities with future binaural CI systems.

Differences in the temporal course of interaural time difference sensitivity between acoustic and electric hearing in amplitude modulated stimuli

Previous studies have shown that normal-hearing (NH) listeners' spatial perception of non-stationary interaural time differences (ITDs) is dominated by the carrier ITD during rising amplitude segments. Here, ITD sensitivity throughout the amplitude-modulation cycle in NH listeners and bilateral cochlear implant (CI) subjects is compared, the latter by means of direct stimulation of a single electrode pair. The data indicate that, while NH listeners are most sensitive to ITDs applied toward the beginning of a modulation cycle at 600 Hz, NH listeners at 200 Hz and especially bilateral CI subjects at 200 pulses per second (pps) are more sensitive to ITDs applied to the modulation maximum. This has implications for spatial-hearing in complex environments: NH listeners' dominant 600-Hz ITD information from the rising amplitude segments comprises direct sound information. The 200-pps low rate required to get ITD sensitivity in CI users results in a higher weight of pulses later in the modulation cycle where the source ITDs are more likely corrupted by reflections. This indirectly indicates that even if future binaural CI processors are able to provide perceptually exploitable ITD information, CI users will likely not get the full benefit from such pulse-based ITD cues in reverberant and other complex environments.

Neural Representation of Interaural Time Differences in Humans-an Objective Measure that Matches Behavioural Performance

Humans, and many other species, exploit small differences in the timing of sounds at the two ears (interaural time difference, ITD) to locate their source and to enhance their detection in background noise. Despite their importance in everyday listening tasks, however, the neural representation of ITDs in human listeners remains poorly understood, and few studies have assessed ITD sensitivity to a similar resolution to that reported perceptually. Here, we report an objective measure of ITD sensitivity in electroencephalography (EEG) signals to abrupt modulations in the interaural phase of amplitude-modulated low-frequency tones. Specifically, we measured following responses to amplitude-modulated sinusoidal signals (520-Hz carrier) in which the stimulus phase at each ear was manipulated to produce discrete interaural phase modulations at minima in the modulation cycle-interaural phase modulation following responses (IPM-FRs). The depth of the interaural phase modulation (IPM) was defined by the sign and the magnitude of the interaural phase difference (IPD) transition which was symmetric around zero. Seven IPM depths were assessed over the range of ±22 ° to ±157 °, corresponding to ITDs largely within the range experienced by human listeners under natural listening conditions (120 to 841 μs). The magnitude of the IPM-FR was maximal for IPM depths in the range of ±67.6 ° to ±112.6 ° and correlated well with performance in a behavioural experiment in which listeners were required to discriminate sounds containing IPMs from those with only static IPDs. The IPM-FR provides a sensitive measure of binaural processing in the human brain and has a potential to assess temporal binaural processing.

Models of the electrically stimulated binaural system: A review

In an increasing number of countries, the standard treatment for deaf individuals is moving toward the implantation of two cochlear implants. Today's device technology and fitting procedure, however, appears as if the two implants would serve two independent ears and brains. Many experimental studies have demonstrated that after careful matching and balancing of left and right stimulation in controlled laboratory studies most patients have almost normal sensitivity to interaural level differences and some sensitivity to interaural time differences (ITDs). Mechanisms underlying the limited ITD sensitivity are still poorly understood and many different aspects may contribute. Recent pioneering computational approaches identified some of the functional implications the electric input imposes on the neural brainstem circuits. Simultaneously these studies have raised new questions and certainly demonstrated that further refinement of the model stages is necessary. They join the experimental study's conclusions that binaural device technology, binaural fitting, specific speech coding strategies, and binaural signal processing algorithms are obviously missing components to maximize the benefit of bilateral implantation. Within this review, the existing models of the electrically stimulated binaural system are explained, compared, and discussed from a viewpoint of a "CI device with auditory system" and from that of neurophysiological research.

Lateralization of stimuli with alternating interaural time differences: The role of monaural envelope cues

A temporally acute binaural system can help to resolve inherent fluctuations in binaural information that are often present in complex auditory scenes. Using a broadband noise stimulus that rapidly alternates between two different values of interaural time difference (ITD), the ability of the binaural system to hear the lateral position resulting from one of the ITD values was investigated. Results show that listeners are able to accurately lateralize brief noise tokens of only 3-7 ms in duration. In two subsequent experiments, the role of an amplitude modulation (AM) imposed on the ITD-switching stimulus used in the first experiment was tested. For wideband stimuli, the temporal position of the ITD target relative to the phase of the AM did not influence absolute lateralization or detection performance. When the stimuli were narrowband, however, detection of the ITD target was best when temporally positioned in the rising portion of the AM. These experiments illustrate that the auditory system is capable of making accurate lateral estimates of very brief moments of ITD information. Furthermore, for these instantaneous changes in ITD information, the stimulus bandwidth can influence the role of envelope cues for the readout of binaural information.

Auditory Model-Based Sound Direction Estimation With Bilateral Cochlear Implants

Users of bilateral cochlear implants (CIs) show above-chance performance in localizing the source of a sound in the azimuthal (horizontal) plane; although localization errors are far worse than for normal-hearing listeners, they are considerably better than for CI listeners with only one implant. In most previous studies, subjects had access to interaural level differences and to interaural time differences conveyed in the temporal envelope. Here, we present a binaural model that predicts the azimuthal direction of sound arrival from a two-channel input signal as it is received at the left and right CI processor. The model includes a replication of a clinical speech-coding strategy, a model of the electrode-nerve interface and binaural brainstem neurons, and three different prediction stages that are trained to map the neural response rate to an azimuthal angle. The model is trained and tested with various noise and speech stimuli created by means of virtual acoustics. Localization error patterns of the model match experimental data and are explicable largely in terms of the nonmonotonic relationship between interaural level difference and azimuthal angle.

Lateralization of noise bursts in interaurally correlated or uncorrelated background noise using interaural level differences

The interaural level difference (ILD) of a lateralized target source may be effectively reduced when the target is presented together with background noise containing zero ILD. It is not certain whether listeners perceive a position congruent with the reduced ILD or the actual target ILD in a lateralization task. Two sets of behavioral experiments revealed that many listeners perceived a position at or even larger than that corresponding to the presented target ILD when a temporal onset/offset asynchrony between the broadband target and the broadband background noise was present. When no temporal asynchrony was present, however, the perceived lateral position indicated a dependency on the coherence of the background noise for several listeners. With interaurally correlated background noise, listeners reported a reduced ILD resulting from the combined target and background noise stimulus. In contrast, several of the listeners made a reasonable estimate of the position corresponding to the target ILD for interaurally uncorrelated, broadband, background noise. No obvious difference in performance was seen between low- or high-frequency stimuli. Extension of a weighting template to the output of a standard equalization-cancellation model was shown to remove a lateral bias on the predicted target ILD resulting from the presence of background noise. Provided that an appropriate weighting template is applied based on knowledge of the background noise coherence, good prediction of the behavioral data is possible.

The influence of pause, attack, and decay duration of the ongoing envelope on sound lateralization

Klein-Hennig et al. [J. Acoust. Soc. Am. 129, 3856-3872 (2011)] introduced a class of high-frequency stimuli for which the envelope shape can be altered by independently varying the attack, hold, decay, and pause durations. These stimuli, originally employed for testing the shape dependence of human listeners' sensitivity to interaural temporal differences (ITDs) in the ongoing envelope, were used to measure the lateralization produced by fixed interaural disparities. Consistent with the threshold ITD data, a steep attack and a non-zero pause facilitate strong ITD-based lateralization. In contrast, those conditions resulted in the smallest interaural level-based lateralization.

Comparing sound localization deficits in bilateral cochlear-implant users and vocoder simulations with normal-hearing listeners

Bilateral cochlear-implant (BiCI) users are less accurate at localizing free-field (FF) sound sources than normal-hearing (NH) listeners. This performance gap is not well understood but is likely due to a combination of compromises in acoustic signal representation by the two independent speech processors and neural degradation of auditory pathways associated with a patient's hearing loss. To exclusively investigate the effect of CI speech encoding on horizontal-plane sound localization, the present study measured sound localization performance in NH subjects listening to vocoder processed and nonvocoded virtual acoustic space (VAS) stimuli. Various aspects of BiCI stimulation such as independently functioning devices, variable across-ear channel selection, and pulsatile stimulation were simulated using uncorrelated noise (Nu), correlated noise (N0), or Gaussian-enveloped tone (GET) carriers during vocoder processing. Additionally, FF sound localization in BiCI users was measured in the same testing environment for comparison. Distinct response patterns across azimuthal locations were evident for both listener groups and were analyzed using a multilevel regression analysis. Simulated implant speech encoding, regardless of carrier, was detrimental to NH localization and the GET vocoder best simulated BiCI FF performance in NH listeners. Overall, the detrimental effect of vocoder processing on NH performance suggests that sound localization deficits may persist even for BiCI patients who have minimal neural degradation associated with their hearing loss and indicates that CI speech encoding plays a significant role in the sound localization deficits experienced by BiCI users.

Emphasis of spatial cues in the temporal fine structure during the rising segments of amplitude-modulated sounds II: single-neuron recordings

Recently, with the use of an amplitude-modulated binaural beat (AMBB), in which sound amplitude and interaural-phase difference (IPD) were modulated with a fixed mutual relationship (Dietz et al. 2013b), we demonstrated that the human auditory system uses interaural timing differences in the temporal fine structure of modulated sounds only during the rising portion of each modulation cycle. However, the degree to which peripheral or central mechanisms contribute to the observed strong dominance of the rising slope remains to be determined. Here, by recording responses of single neurons in the medial superior olive (MSO) of anesthetized gerbils and in the inferior colliculus (IC) of anesthetized guinea pigs to AMBBs, we report a correlation between the position within the amplitude-modulation (AM) cycle generating the maximum response rate and the position at which the instantaneous IPD dominates the total neural response. The IPD during the rising segment dominates the total response in 78% of MSO neurons and 69% of IC neurons, with responses of the remaining neurons predominantly coding the IPD around the modulation maximum. The observed diversity of dominance regions within the AM cycle, especially in the IC, and its comparison with the human behavioral data suggest that only the subpopulation of neurons with rising slope dominance codes the sound-source location in complex listening conditions. A comparison of two models to account for the data suggests that emphasis on IPDs during the rising slope of the AM cycle depends on adaptation processes occurring before binaural interaction.

Lateralization based on interaural differences in the second-order amplitude modulator

Second-order amplitude modulation is a relatively slow variation of the modulation depth of a first-order amplitude modulation with higher frequency. In contrast to first-order modulation, which appears as a physical component in the stimulus spectrum after half-wave rectification, second-order modulation is not necessarily demodulated by the auditory periphery. For binaural processing of second-order amplitude modulated stimuli it is unknown whether interaural time differences (ITDs) in the second-order modulation result in a lateralized percept. Thus, second-order modulation can serve as a tool to investigate whether demodulation of interaurally delayed components is a prerequisite for lateralization. In most of the psychoacoustic experiments presented here, a 25 Hz sinusoidally amplitude-modulated (SAM) 160 Hz tone was either transposed to 4 kHz by half-wave rectifying this SAM waveform before multiplication with a 4 kHz tone (TSAM), or by adding an offset before multiplication (SAMAM). The experiments revealed an inability to lateralize the SAMAM based on ITDs in the 25 Hz component, whereas subjects could lateralize the TSAM. Given that only the TSAM results in a demodulated 25 Hz component after peripheral auditory processing, this result supports the hypothesis that demodulation is a prerequisite for lateralization, which has consequences for temporal modulation processing in models of binaural interaction.

Perceptual sensitivity to high-frequency interaural time differences created by rustling sounds

Interaural time differences (ITDs) can be used to localize sounds in the horizontal plane. ITDs can be extracted from either the fine structure of low-frequency sounds or from the envelopes of high-frequency sounds. Studies of the latter have included stimuli with periodic envelopes like amplitude-modulated tones or transposed stimuli, and high-pass filtered Gaussian noises. Here, four experiments are presented investigating the perceptual relevance of ITD cues in synthetic and recorded "rustling" sounds. Both share the broad long-term power spectrum with Gaussian noise but provide more pronounced envelope fluctuations than Gaussian noise, quantified by an increased waveform fourth moment, W. The current data show that the JNDs in ITD for band-pass rustling sounds tended to improve with increasing W and with increasing bandwidth when the sounds were band limited. In contrast, no influence of W on JND was observed for broadband sounds, apparently because of listeners' sensitivity to ITD in low-frequency fine structure, present in the broadband sounds. Second, it is shown that for high-frequency rustling sounds ITD JNDs can be as low as 30 μs. The third result was that the amount of dominance for ITD extraction of low frequencies decreases systematically with increasing amount of envelope fluctuations. Finally, it is shown that despite the exceptionally good envelope ITD sensitivity evident with high-frequency rustling sounds, minimum audible angles of both synthetic and recorded high-frequency rustling sounds in virtual acoustic space are still best when the angular information is mediated by interaural level differences.

The influence of different segments of the ongoing envelope on sensitivity to interaural time delays

The auditory system is sensitive to interaural timing disparities in the fine structure and the envelope of sounds, each contributing important cues for lateralization. In this study, psychophysical measurements were conducted with customized envelope waveforms in order to investigate the isolated effect of different segments of a periodic, ongoing envelope on lateralization. One envelope cycle was composed of the four segments attack flank, hold duration, decay flank, and pause duration, which were independently varied to customize the envelope waveform. The envelope waveforms were applied to a 4-kHz sinusoidal carrier, and just noticeable envelope interaural time differences were measured in six normal hearing subjects. The results indicate that attack durations and pause durations prior to the attack are the most important stimulus characteristics for processing envelope timing disparities. The results were compared to predictions of three binaural lateralization models based on the normalized cross correlation coefficient. Two of the models included an additional stage to mimic neural adaptation prior to binaural interaction, involving either a single short time constant (5 ms) or a combination of five time constants up to 500 ms. It was shown that the model with the single short time constant accounted best for the data.

Improvements of sound localization abilities by the facial ruff of the barn owl (Tyto alba) as demonstrated by virtual ruff removal

Background

When sound arrives at the eardrum it has already been filtered by the body, head, and outer ear. This process is mathematically described by the head-related transfer functions (HRTFs), which are characteristic for the spatial position of a sound source and for the individual ear. HRTFs in the barn owl (Tyto alba) are also shaped by the facial ruff, a specialization that alters interaural time differences (ITD), interaural intensity differences (ILD), and the frequency spectrum of the incoming sound to improve sound localization. Here we created novel stimuli to simulate the removal of the barn owl's ruff in a virtual acoustic environment, thus creating a situation similar to passive listening in other animals, and used these stimuli in behavioral tests.

Methodology/Principal Findings

HRTFs were recorded from an owl before and after removal of the ruff feathers. Normal and ruff-removed conditions were created by filtering broadband noise with the HRTFs. Under normal virtual conditions, no differences in azimuthal head-turning behavior between individualized and non-individualized HRTFs were observed. The owls were able to respond differently to stimuli from the back than to stimuli from the front having the same ITD. By contrast, such a discrimination was not possible after the virtual removal of the ruff. Elevational head-turn angles were (slightly) smaller with non-individualized than with individualized HRTFs. The removal of the ruff resulted in a large decrease in elevational head-turning amplitudes.

Conclusions/Significance

The facial ruff a) improves azimuthal sound localization by increasing the ITD range and b) improves elevational sound localization in the frontal field by introducing a shift of iso–ILD lines out of the midsagittal plane, which causes ILDs to increase with increasing stimulus elevation. The changes at the behavioral level could be related to the changes in the binaural physical parameters that occurred after the virtual removal of the ruff. These data provide new insights into the function of external hearing structures and open up the possibility to apply the results on autonomous agents, creation of virtual auditory environments for humans, or in hearing aids.