Emphasis of spatial cues in the temporal fine structure during the rising segments of amplitude-modulated sounds II: single-neuron recordings

Noises on-How the Brain Deals with Acoustic Noise

What is noise? When does a sound form part of the acoustic background and when might it come to our attention as part of the foreground? Our brain seems to filter out irrelevant sounds in a seemingly effortless process, but how this is achieved remains opaque and, to date, unparalleled by any algorithm. In this review, we discuss how noise can be both background and foreground, depending on what a listener/brain is trying to achieve. We do so by addressing questions concerning the brain's potential bias to interpret certain sounds as part of the background, the extent to which the interpretation of sounds depends on the context in which they are heard, as well as their ethological relevance, task-dependence, and a listener's overall mental state. We explore these questions with specific regard to the implicit, or statistical, learning of sounds and the role of feedback loops between cortical and subcortical auditory structures.

Adaptation in auditory processing

Adaptation is an essential feature of auditory neurons, which reduces their responses to unchanging and recurring sounds and allows their response properties to be matched to the constantly changing statistics of sounds that reach the ears. As a consequence, processing in the auditory system highlights novel or unpredictable sounds and produces an efficient representation of the vast range of sounds that animals can perceive by continually adjusting the sensitivity and, to a lesser extent, the tuning properties of neurons to the most commonly encountered stimulus values. Together with attentional modulation, adaptation to sound statistics also helps to generate neural representations of sound that are tolerant to background noise and therefore plays a vital role in auditory scene analysis. In this review, we consider the diverse forms of adaptation that are found in the auditory system in terms of the processing levels at which they arise, the underlying neural mechanisms, and their impact on neural coding and perception. We also ask what the dynamics of adaptation, which can occur over multiple timescales, reveal about the statistical properties of the environment. Finally, we examine how adaptation to sound statistics is influenced by learning and experience and changes as a result of aging and hearing loss.

Temporal integration of monaural and dichotic frequency modulation

Frequency modulation (FM) detection at low modulation frequencies is commonly used as an index of temporal fine-structure processing. The present study evaluated the rate of improvement in monaural and dichotic FM across a range of test parameters. In experiment I, dichotic and monaural FM detection was measured as a function of duration and modulator starting phase. Dichotic FM thresholds were lower than monaural FM thresholds and the modulator starting phase had no effect on detection. Experiment II measured monaural FM detection for signals that differed in modulation rate and duration such that the improvement with duration in seconds (carrier) or cycles (modulator) was compared. Monaural FM detection improved monotonically with the number of modulation cycles, suggesting that the modulator is extracted prior to detection. Experiment III measured dichotic FM detection for shorter signal durations to test the hypothesis that dichotic FM relies primarily on the signal onset. The rate of improvement decreased as duration increased, which is consistent with the use of primarily onset cues for the detection of dichotic FM. These results establish that improvement with duration occurs as a function of the modulation cycles at a rate consistent with the independent-samples model for monaural FM, but later cycles contribute less to detection in dichotic FM.

Physiological Diversity Influences Detection of Stimulus Envelope and Fine Structure in Neurons of the Medial Superior Olive

The neurons of the medial superior olive (MSO) of mammals extract azimuthal information from the delays between sounds reaching the two ears [interaural time differences (ITDs)]. Traditionally, all models of sound localization have assumed that MSO neurons represent a single population of cells with specialized and homogeneous intrinsic and synaptic properties that enable the detection of synaptic coincidence on a timescale of tens to hundreds of microseconds. Here, using patch-clamp recordings from large populations of anatomically labeled neurons in brainstem slices from male and female Mongolian gerbils (Meriones unguiculatus), we show that MSO neurons are far more physiologically diverse than previously appreciated, with properties that depend regionally on cell position along the topographic map of frequency. Despite exhibiting a similar morphology, neurons in the MSO exhibit subthreshold oscillations of differing magnitudes that drive action potentials at rates between 100 and 800 Hz. These oscillations are driven primarily by voltage-gated sodium channels and are distinct from resonant properties derived from other active membrane properties. We show that graded differences in these and other physiological properties across the MSO neuron population enable the MSO to duplex the encoding of ITD information in both fast, submillisecond time-varying signals as well as in slower envelopes.SIGNIFICANCE STATEMENT Neurons in the medial superior olive (MSO) encode sound localization cues by detecting microsecond differences in the arrival times of inputs from the left and right ears, and it has been assumed that this computation is made possible by highly stereotyped structural and physiological specializations. Here we report using a large (>400) sample size in which MSO neurons show a strikingly large continuum of functional properties despite exhibiting similar morphologies. We demonstrate that subthreshold oscillations mediated by voltage-gated Na+ channels play a key role in conferring graded differences in firing properties. This functional diversity likely confers capabilities of processing both fast, submillisecond-scale synaptic activity (acoustic "fine structure"), and slow-rising envelope information that is found in amplitude-modulated sounds and speech patterns.

The influence of envelope shape on the lateralization of amplitude-modulated, low-frequency sound

For abruptly gated sound, interaural time difference (ITD) cues at onset carry greater perceptual weight than those following. This research explored how envelope shape influences such carrier ITD weighting. Experiment 1 assessed the perceived lateralization of a tonal binaural beat that transitioned through ITD (diotic envelope, mean carrier frequency of 500 Hz). Listeners' left/right lateralization judgments were compared to those for static-ITD tones. For an 8 Hz sinusoidally amplitude-modulated envelope, ITD cues 24 ms after onset well-predicted reported sidedness. For an equivalent-duration "abrupt" envelope, which was unmodulated besides 20-ms onset/offset ramps, reported sidedness corresponded to ITDs near onset (e.g., 6 ms). However, unlike for sinusoidal amplitude modulation, ITDs toward offset seemingly also influenced perceived sidedness. Experiment 2 adjusted the duration of the offset ramp (25-75 ms) and found evidence for such offset weighting only for the most abrupt ramp tested. In experiment 3, an ITD was imposed on a brief segment of otherwise diotic filtered noise. Listeners discriminated right- from left-leading ITDs. In sinusoidal amplitude modulation, thresholds were lowest when the ITD segment occurred during rising amplitude. For the abrupt envelope, the lowest thresholds were observed when the segment occurred at either onset or offset. These experiments demonstrate the influence of envelope profile on carrier ITD sensitivity.

Auditory Brainstem Models: Adapting Cochlear Nuclei Improve Spatial Encoding by the Medial Superior Olive in Reverberation

Listeners typically perceive a sound as originating from the direction of its source, even as direct sound is followed milliseconds later by reflected sound from multiple different directions. Early-arriving sound is emphasised in the ascending auditory pathway, including the medial superior olive (MSO) where binaural neurons encode the interaural-time-difference (ITD) cue for spatial location. Perceptually, weighting of ITD conveyed during rising sound energy is stronger at 600 Hz than at 200 Hz, consistent with the minimum stimulus rate for binaural adaptation, and with the longer reverberation times at 600 Hz, compared with 200 Hz, in many natural outdoor environments. Here, we computationally explore the combined efficacy of adaptation prior to the binaural encoding of ITD cues, and excitatory binaural coincidence detection within MSO neurons, in emphasising ITDs conveyed in early-arriving sound. With excitatory inputs from adapting, nonlinear model spherical bushy cells (SBCs) of the bilateral cochlear nuclei, a nonlinear model MSO neuron with low-threshold potassium channels reproduces the rate-dependent emphasis of rising vs. peak sound energy in ITD encoding; adaptation is equally effective in the model MSO. Maintaining adaptation in model SBCs, and adjusting membrane speed in model MSO neurons, 'left' and 'right' populations of computationally efficient, linear model SBCs and MSO neurons reproduce this stronger weighting of ITD conveyed during rising sound energy at 600 Hz compared to 200 Hz. This hemispheric population model demonstrates a link between strong weighting of spatial information during rising sound energy, and correct unambiguous lateralisation of a speech source in reverberation.

Estimating the perceptual weighting of interaural time difference cues in amplitude modulated binaural beats

For an abruptly gated sound, perceived lateralization is determined primarily by binaural cues at onset. Relatively less is known about the temporal weighing of binaural cues-such as interaural time difference (ITD)-during more naturalistic modulation profiles. Here, an experiment measured the lateralization of a tonal binaural beat modulated by a diotic, 8-Hz sinusoidal amplitude modulation. Binaural beat lateralization (left/right, two alternatives) was compared to that for tones with static ITDs. Across three mean carrier frequencies (200, 500, and 800 Hz), ITDs occurring during early rising amplitude (e.g., 20-25 ms after onset) predicted the perceived lateralization of the binaural beat signals well.

Neural coding and perception of auditory motion direction based on interaural time differences

While motion is important for parsing a complex auditory scene into perceptual objects, how it is encoded in the auditory system is unclear. Perceptual studies suggest that the ability to identify the direction of motion is limited by the duration of the moving sound, yet we can detect changes in interaural differences at even shorter durations. To understand the source of these distinct temporal limits, we recorded from single units in the inferior colliculus (IC) of unanesthetized rabbits in response to noise stimuli containing a brief segment with linearly time-varying interaural time difference ("ITD sweep") temporally embedded in interaurally uncorrelated noise. We also tested the ability of human listeners to either detect the ITD sweeps or identify the motion direction. Using a point-process model to separate the contributions of stimulus dependence and spiking history to single-neuron responses, we found that the neurons respond primarily by following the instantaneous ITD rather than exhibiting true direction selectivity. Furthermore, using an optimal classifier to decode the single-neuron responses, we found that neural threshold durations of ITD sweeps for both direction identification and detection overlapped with human threshold durations even though the average response of the neurons could track the instantaneous ITD beyond psychophysical limits. Our results suggest that the IC does not explicitly encode motion direction, but internal neural noise may limit the speed at which we can identify the direction of motion.NEW & NOTEWORTHY Recognizing motion and identifying an object's trajectory are important for parsing a complex auditory scene, but how we do so is unclear. We show that neurons in the auditory midbrain do not exhibit direction selectivity as found in the visual system but instead follow the trajectory of the motion in their temporal firing patterns. Our results suggest that the inherent variability in neural firings may limit our ability to identify motion direction at short durations.

The impact of peripheral mechanisms on the precedence effect

When two similar sounds are presented from different locations, with one (the lead) preceding the other (the lag) by a small delay, listeners typically report hearing one sound near the location of the lead sound source-this is called the precedence effect (PE). Several questions about the underlying mechanisms that produce the PE are asked. (1) How might listeners' relative weighting of cues at onset versus ongoing stimulus portions affect perceived lateral position of long-duration lead/lag noise stimuli? (2) What are the factors that influence this weighting? (3) Are the mechanisms invoked to explain the PE for transient stimuli applicable to long-duration stimuli? To answer these questions, lead/lag noise stimuli are presented with a range of durations, onset slopes, and lag-to-lead level ratios over headphones. Monaural, peripheral mechanisms, and binaural cue extraction are modeled to estimate the cues available for determination of perceived laterality. Results showed that all three stimulus manipulations affect the relative weighting of onset and ongoing cues and that mechanisms invoked to explain the PE for transient stimuli are also applicable to the PE, in terms of both onset and ongoing segments of long-duration, lead/lag stimuli.

Effects of age on sensitivity to interaural time differences in envelope and fine structure, individually and in combination

Sensitivity to interaural time differences (ITDs) in envelope and temporal fine structure (TFS) of amplitude-modulated (AM) tones was assessed for young and older subjects, all with clinically normal hearing at the carrier frequencies of 250 and 500 Hz. Some subjects had hearing loss at higher frequencies. In experiment 1, thresholds for detecting changes in ITD were measured when the ITD was present in the TFS alone (ITD_TFS), the envelope alone (ITD_ENV), or both (ITD_TFS/ENV). Thresholds tended to be higher for the older than for the young subjects. ITD_ENV thresholds were much higher than ITD_TFS thresholds, while ITD_TFS/ENV thresholds were similar to ITD_TFS thresholds. ITD_TFS thresholds were lower than ITD thresholds obtained with an unmodulated pure tone, indicating that uninformative AM can improve ITD_TFS discrimination. In experiment 2, equally detectable values of ITD_TFS and ITD_ENV were combined so as to give consistent or inconsistent lateralization. There were large individual differences, but several subjects gave scores that were much higher than would be expected from the optimal combination of independent sources of information, even for the inconsistent condition. It is suggested that ITD_TFS and ITD_ENV cues are processed partly independently, but that both cues influence lateralization judgments, even when one cue is uninformative.

Can monaural temporal masking explain the ongoing precedence effect?

The precedence effect for transient sounds has been proposed to be based primarily on monaural processes, manifested by asymmetric temporal masking. This study explored the potential for monaural explanations with longer ("ongoing") sounds exhibiting the precedence effect. Transient stimuli were single lead-lag noise burst pairs; ongoing stimuli were trains of 63 burst pairs. Unlike with transients, monaural masking data for ongoing sounds showed no advantage for the lead, and are inconsistent with asymmetric audibility as an explanation for ongoing precedence. This result, along with supplementary measurements of interaural time discrimination, suggests different explanations for transient and ongoing precedence.

A framework for testing and comparing binaural models

Auditory research has a rich history of combining experimental evidence with computational simulations of auditory processing in order to deepen our theoretical understanding of how sound is processed in the ears and in the brain. Despite significant progress in the amount of detail and breadth covered by auditory models, for many components of the auditory pathway there are still different model approaches that are often not equivalent but rather in conflict with each other. Similarly, some experimental studies yield conflicting results which has led to controversies. This can be best resolved by a systematic comparison of multiple experimental data sets and model approaches. Binaural processing is a prominent example of how the development of quantitative theories can advance our understanding of the phenomena, but there remain several unresolved questions for which competing model approaches exist. This article discusses a number of current unresolved or disputed issues in binaural modelling, as well as some of the significant challenges in comparing binaural models with each other and with the experimental data. We introduce an auditory model framework, which we believe can become a useful infrastructure for resolving some of the current controversies. It operates models over the same paradigms that are used experimentally. The core of the proposed framework is an interface that connects three components irrespective of their underlying programming language: The experiment software, an auditory pathway model, and task-dependent decision stages called artificial observers that provide the same output format as the test subject.

Influence of envelope waveform on ITD sensitivity of neurons in the auditory midbrain

Using single-neuron electrophysiology, we show that the precise shape of a sound’s “energy envelope” is a critical factor in determining how well midbrain neurons are able to convey information about auditory spatial cues. Consistent with human behavioral performance, sounds with rapidly rising energy and relatively long intervals between energy bursts are best at conveying spatial information. The data suggest specific sound energy patterns that might best be applied to hearing devices to aid spatial listening.

Interaural time differences (ITDs) conveyed by the modulated envelopes of high-frequency sounds can serve as a cue for localizing a sound source. Klein-Hennig et al. (J Acoust Soc Am 129: 3856, 2011) demonstrated the envelope attack (the rate at which stimulus energy in the envelope increases) and the duration of the pause (the interval between successive envelope pulses) as important factors affecting sensitivity to envelope ITDs in human listeners. Modulated sounds with rapid attacks and long pauses produce the lowest ITD discrimination thresholds. The duration of the envelope’s sustained component (sustain) and the rate at which stimulus energy falls at the offset of the envelope (decay) are only minor factors. We assessed the responses of 71 single neurons, recorded from the midbrains of 15 urethane-anesthetized tri-colored guinea pigs, to envelope shapes in which the four envelope components, i.e., attack, sustain, decay, and pause, were systematically varied. We confirmed the importance of the attack and pause components in generating ITD-sensitive responses. Analysis of neural firing rates demonstrated more neurons (49/71) show ITD sensitivity in response to “damped” stimuli (fast attack and slow decay) compared with “ramped” stimuli (slow attack and fast decay) (14/71). Furthermore, the lowest threshold for the damped stimulus (91 μs) was lower by a factor of 4 than that for the temporally reversed ramped envelope shape (407 μs). The data confirm the importance of fast attacks and optimal pause durations in generating sensitivity to ITDs conveyed in the modulated envelopes of high-frequency sounds and are incompatible with models of ITD processing based on the integration of sound energy over time.

NEW & NOTEWORTHY Using single-neuron electrophysiology, we show that the precise shape of a sound’s “energy envelope” is a critical factor in determining how well midbrain neurons are able to convey information about auditory spatial cues. Consistent with human behavioral performance, sounds with rapidly rising energy and relatively long intervals between energy bursts are best at conveying spatial information. The data suggest specific sound energy patterns that might best be applied to hearing devices to aid spatial listening.

Neural coding of time-varying interaural time differences and time-varying amplitude in the inferior colliculus

Binaural cues occurring in natural environments are frequently time varying, either from the motion of a sound source or through interactions between the cues produced by multiple sources. Yet, a broad understanding of how the auditory system processes dynamic binaural cues is still lacking. In the current study, we directly compared neural responses in the inferior colliculus (IC) of unanesthetized rabbits to broadband noise with time-varying interaural time differences (ITD) with responses to noise with sinusoidal amplitude modulation (SAM) over a wide range of modulation frequencies. On the basis of prior research, we hypothesized that the IC, one of the first stages to exhibit tuning of firing rate to modulation frequency, might use a common mechanism to encode time-varying information in general. Instead, we found weaker temporal coding for dynamic ITD compared with amplitude modulation and stronger effects of adaptation for amplitude modulation. The differences in temporal coding of dynamic ITD compared with SAM at the single-neuron level could be a neural correlate of "binaural sluggishness," the inability to perceive fluctuations in time-varying binaural cues at high modulation frequencies, for which a physiological explanation has so far remained elusive. At ITD-variation frequencies of 64 Hz and above, where a temporal code was less effective, noise with a dynamic ITD could still be distinguished from noise with a constant ITD through differences in average firing rate in many neurons, suggesting a frequency-dependent tradeoff between rate and temporal coding of time-varying binaural information.NEW & NOTEWORTHY Humans use time-varying binaural cues to parse auditory scenes comprising multiple sound sources and reverberation. However, the neural mechanisms for doing so are poorly understood. Our results demonstrate a potential neural correlate for the reduced detectability of fluctuations in time-varying binaural information at high speeds, as occurs in reverberation. The results also suggest that the neural mechanisms for processing time-varying binaural and monaural cues are largely distinct.

Input timing for spatial processing is precisely tuned via constant synaptic delays and myelination patterns in the auditory brainstem

Precise timing of synaptic inputs is a fundamental principle of neural circuit processing. The temporal precision of postsynaptic input integration is known to vary with the computational requirements of a circuit, yet how the timing of action potentials is tuned presynaptically to match these processing demands is not well understood. In particular, action potential timing is shaped by the axonal conduction velocity and the duration of synaptic transmission delays within a pathway. However, it is not known to what extent these factors are adapted to the functional constraints of the respective circuit. Here, we report the finding of activity-invariant synaptic transmission delays as a functional adaptation for input timing adjustment in a brainstem sound localization circuit. We compared axonal and synaptic properties of the same pathway between two species with dissimilar timing requirements (gerbil and mouse): In gerbils (like humans), neuronal processing of sound source location requires exceptionally high input precision in the range of microseconds, but not in mice. Activity-invariant synaptic transmission and conduction delays were present exclusively in fast conducting axons of gerbils that also exhibited unusual structural adaptations in axon myelination for increased conduction velocity. In contrast, synaptic transmission delays in mice varied depending on activity levels, and axonal myelination and conduction velocity exhibited no adaptations. Thus, the specializations in gerbils and their absence in mice suggest an optimization of axonal and synaptic properties to the specific demands of sound localization. These findings significantly advance our understanding of structural and functional adaptations for circuit processing.

Exploring binaural hearing in gerbils (Meriones unguiculatus) using virtual headphones

The Mongolian gerbil (Meriones unguiculatus) has become a key species in investigations of the neural processing of sound localization cues in mammals. While its sound localization has been tested extensively under free-field stimulation, many neurophysiological studies use headphones to present signals with binaural localization cues. The gerbil's behavioral sensitivity to binaural cues, however, is unknown for the lack of appropriate stimulation paradigms in awake behaving gerbils. We close this gap in knowledge by mimicking a headphone stimulation; we use free-field loudspeakers and apply cross-talk cancellation techniques to present pure tones with binaural cues via “virtual headphones” to gerbils trained in a sound localization task. All gerbils were able to lateralize sounds depending on the interaural time or level difference (ITD and ILD, respectively). For ITD stimuli, reliable responses were seen for frequencies ≤2.9 kHz, the highest frequency tested with ITD stimuli. ITD sensitivity was frequency-dependent with the highest sensitivity observed at 1 kHz. For stimuli with ITD outside the gerbil's physiological range, responses were cyclic indicating the use of phase information when lateralizing narrow-band sounds. For ILD stimuli, reliable responses were obtained for frequencies ≥2 kHz. The comparison of ITD and ILD thresholds with ITD and ILD thresholds derived from gerbils’ free-field performance suggests that ongoing ITD information is the main cue for sound localization at frequencies <2 kHz. At 2 kHz, ITD and ILD cues are likely used in a complementary way. Verification of the use of the virtual headphones suggests that they can serve as a suitable substitute for conventional headphones particularly at frequencies ≤2 kHz.

Differences in the temporal course of interaural time difference sensitivity between acoustic and electric hearing in amplitude modulated stimuli

Previous studies have shown that normal-hearing (NH) listeners' spatial perception of non-stationary interaural time differences (ITDs) is dominated by the carrier ITD during rising amplitude segments. Here, ITD sensitivity throughout the amplitude-modulation cycle in NH listeners and bilateral cochlear implant (CI) subjects is compared, the latter by means of direct stimulation of a single electrode pair. The data indicate that, while NH listeners are most sensitive to ITDs applied toward the beginning of a modulation cycle at 600 Hz, NH listeners at 200 Hz and especially bilateral CI subjects at 200 pulses per second (pps) are more sensitive to ITDs applied to the modulation maximum. This has implications for spatial-hearing in complex environments: NH listeners' dominant 600-Hz ITD information from the rising amplitude segments comprises direct sound information. The 200-pps low rate required to get ITD sensitivity in CI users results in a higher weight of pulses later in the modulation cycle where the source ITDs are more likely corrupted by reflections. This indirectly indicates that even if future binaural CI processors are able to provide perceptually exploitable ITD information, CI users will likely not get the full benefit from such pulse-based ITD cues in reverberant and other complex environments.

Evaluation of a method for enhancing interaural level differences at low frequencies

A method (called binaural enhancement) for enhancing interaural level differences at low frequencies, based on estimates of interaural time differences, was developed and evaluated. Five conditions were compared, all using simulated hearing-aid processing: (1) Linear amplification with frequency-response shaping; (2) binaural enhancement combined with linear amplification and frequency-response shaping; (3) slow-acting four-channel amplitude compression with independent compression at the two ears (AGC4CH); (4) binaural enhancement combined with four-channel compression (BE-AGC4CH); and (5) four-channel compression but with the compression gains synchronized across ears. Ten hearing-impaired listeners were tested, and gains and compression ratios for each listener were set to match targets prescribed by the CAM2 fitting method. Stimuli were presented via headphones, using virtualization methods to simulate listening in a moderately reverberant room. The intelligibility of speech at ±60° azimuth in the presence of competing speech on the opposite side of the head at ±60° azimuth was not affected by the binaural enhancement processing. Sound localization was significantly better for condition BE-AGC4CH than for condition AGC4CH for a sentence, but not for broadband noise, lowpass noise, or lowpass amplitude-modulated noise. The results suggest that the binaural enhancement processing can improve localization for sounds with distinct envelope fluctuations.

Physiology and anatomy of neurons in the medial superior olive of the mouse

In mammals with good low-frequency hearing, the medial superior olive (MSO) computes sound location by comparing differences in the arrival time of a sound at each ear, called interaural time disparities (ITDs). Low-frequency sounds are not reflected by the head, and therefore level differences and spectral cues are minimal or absent, leaving ITDs as the only cue for sound localization. Although mammals with high-frequency hearing and small heads (e.g., bats, mice) barely experience ITDs, the MSO is still present in these animals. Yet, aside from studies in specialized bats, in which the MSO appears to serve functions other than ITD processing, it has not been studied in small mammals that do not hear low frequencies. Here we describe neurons in the mouse brain stem that share prominent anatomical, morphological, and physiological properties with the MSO in species known to use ITDs for sound localization. However, these neurons also deviate in some important aspects from the typical MSO, including a less refined arrangement of cell bodies, dendrites, and synaptic inputs. In vitro, the vast majority of neurons exhibited a single, onset action potential in response to suprathreshold depolarization. This spiking pattern is typical of MSO neurons in other species and is generated from a complement of K_v1, K_v3, and I_H currents. In vivo, mouse MSO neurons show bilateral excitatory and inhibitory tuning as well as an improvement in temporal acuity of spiking during bilateral acoustic stimulation. The combination of classical MSO features like those observed in gerbils with more unique features similar to those observed in bats and opossums make the mouse MSO an interesting model for exploiting genetic tools to test hypotheses about the molecular mechanisms and evolution of ITD processing.

Models of the electrically stimulated binaural system: A review

In an increasing number of countries, the standard treatment for deaf individuals is moving toward the implantation of two cochlear implants. Today's device technology and fitting procedure, however, appears as if the two implants would serve two independent ears and brains. Many experimental studies have demonstrated that after careful matching and balancing of left and right stimulation in controlled laboratory studies most patients have almost normal sensitivity to interaural level differences and some sensitivity to interaural time differences (ITDs). Mechanisms underlying the limited ITD sensitivity are still poorly understood and many different aspects may contribute. Recent pioneering computational approaches identified some of the functional implications the electric input imposes on the neural brainstem circuits. Simultaneously these studies have raised new questions and certainly demonstrated that further refinement of the model stages is necessary. They join the experimental study's conclusions that binaural device technology, binaural fitting, specific speech coding strategies, and binaural signal processing algorithms are obviously missing components to maximize the benefit of bilateral implantation. Within this review, the existing models of the electrically stimulated binaural system are explained, compared, and discussed from a viewpoint of a "CI device with auditory system" and from that of neurophysiological research.

Sensitivity to Interaural Time Differences Conveyed in the Stimulus Envelope: Estimating Inputs of Binaural Neurons Through the Temporal Analysis of Spike Trains

Sound-source localization in the horizontal plane relies on detecting small differences in the timing and level of the sound at the two ears, including differences in the timing of the modulated envelopes of high-frequency sounds (envelope interaural time differences (ITDs)). We investigated responses of single neurons in the inferior colliculus (IC) to a wide range of envelope ITDs and stimulus envelope shapes. By a novel means of visualizing neural activity relative to different portions of the periodic stimulus envelope at each ear, we demonstrate the role of neuron-specific excitatory and inhibitory inputs in creating ITD sensitivity (or the lack of it) depending on the specific shape of the stimulus envelope. The underlying binaural brain circuitry and synaptic parameters were modeled individually for each neuron to account for neuron-specific activity patterns. The model explains the effects of envelope shapes on sensitivity to envelope ITDs observed in both normal-hearing listeners and in neural data, and has consequences for understanding how ITD information in stimulus envelopes might be maximized in users of bilateral cochlear implants-for whom ITDs conveyed in the stimulus envelope are the only ITD cues available.

Lateralization of stimuli with alternating interaural time differences: The role of monaural envelope cues

A temporally acute binaural system can help to resolve inherent fluctuations in binaural information that are often present in complex auditory scenes. Using a broadband noise stimulus that rapidly alternates between two different values of interaural time difference (ITD), the ability of the binaural system to hear the lateral position resulting from one of the ITD values was investigated. Results show that listeners are able to accurately lateralize brief noise tokens of only 3-7 ms in duration. In two subsequent experiments, the role of an amplitude modulation (AM) imposed on the ITD-switching stimulus used in the first experiment was tested. For wideband stimuli, the temporal position of the ITD target relative to the phase of the AM did not influence absolute lateralization or detection performance. When the stimuli were narrowband, however, detection of the ITD target was best when temporally positioned in the rising portion of the AM. These experiments illustrate that the auditory system is capable of making accurate lateral estimates of very brief moments of ITD information. Furthermore, for these instantaneous changes in ITD information, the stimulus bandwidth can influence the role of envelope cues for the readout of binaural information.

Sensitivity to Envelope Interaural Time Differences at High Modulation Rates

Sensitivity to interaural time differences (ITDs) conveyed in the temporal fine structure of low-frequency tones and the modulated envelopes of high-frequency sounds are considered comparable, particularly for envelopes shaped to transmit similar fidelity of temporal information normally present for low-frequency sounds. Nevertheless, discrimination performance for envelope modulation rates above a few hundred Hertz is reported to be poor-to the point of discrimination thresholds being unattainable-compared with the much higher (>1,000 Hz) limit for low-frequency ITD sensitivity, suggesting the presence of a low-pass filter in the envelope domain. Further, performance for identical modulation rates appears to decline with increasing carrier frequency, supporting the view that the low-pass characteristics observed for envelope ITD processing is carrier-frequency dependent. Here, we assessed listeners' sensitivity to ITDs conveyed in pure tones and in the modulated envelopes of high-frequency tones. ITD discrimination for the modulated high-frequency tones was measured as a function of both modulation rate and carrier frequency. Some well-trained listeners appear able to discriminate ITDs extremely well, even at modulation rates well beyond 500 Hz, for 4-kHz carriers. For one listener, thresholds were even obtained for a modulation rate of 800 Hz. The highest modulation rate for which thresholds could be obtained declined with increasing carrier frequency for all listeners. At 10 kHz, the highest modulation rate at which thresholds could be obtained was 600 Hz. The upper limit of sensitivity to ITDs conveyed in the envelope of high-frequency modulated sounds appears to be higher than previously considered.

A Neural Model of Auditory Space Compatible with Human Perception under Simulated Echoic Conditions

In a typical auditory scene, sounds from different sources and reflective surfaces summate in the ears, causing spatial cues to fluctuate. Prevailing hypotheses of how spatial locations may be encoded and represented across auditory neurons generally disregard these fluctuations and must therefore invoke additional mechanisms for detecting and representing them. Here, we consider a different hypothesis in which spatial perception corresponds to an intermediate or sub-maximal firing probability across spatially selective neurons within each hemisphere. The precedence or Haas effect presents an ideal opportunity for examining this hypothesis, since the temporal superposition of an acoustical reflection with sounds arriving directly from a source can cause otherwise stable cues to fluctuate. Our findings suggest that subjects’ experiences may simply reflect the spatial cues that momentarily arise under various acoustical conditions and how these cues are represented. We further suggest that auditory objects may acquire “edges” under conditions when interaural time differences are broadly distributed.

Temporal weighting of binaural information at low frequencies: Discrimination of dynamic interaural time and level differences

The importance of sound onsets in binaural hearing has been addressed in many studies, particularly at high frequencies, where the onset of the envelope may carry much of the useful binaural information. Some studies suggest that sound onsets might play a similar role in the processing of binaural cues [e.g., fine-structure interaural time differences (ITD)] at low frequencies. This study measured listeners' sensitivity to ITD and interaural level differences (ILD) present in early (i.e., onset) and late parts of 80-ms pure tones of 250-, 500-, and 1000-Hz frequency. Following previous studies, tones carried static interaural cues or dynamic cues that peaked at sound onset and diminished to zero at sound offset or vice versa. Although better thresholds were observed in static than dynamic conditions overall, ITD discrimination was especially impaired, regardless of frequency, when cues were not available at sound onset. Results for ILD followed a similar pattern at 1000 Hz; at lower frequencies, ILD thresholds did not differ significantly between dynamic-cue conditions. The results support the "onset" hypothesis of Houtgast and Plomp [(1968). J. Acoust. Soc. Am. 44, 807-812] for ITD discrimination, but not necessarily ILD discrimination, in low-frequency pure tones.

The influence of pause, attack, and decay duration of the ongoing envelope on sound lateralization

Klein-Hennig et al. [J. Acoust. Soc. Am. 129, 3856-3872 (2011)] introduced a class of high-frequency stimuli for which the envelope shape can be altered by independently varying the attack, hold, decay, and pause durations. These stimuli, originally employed for testing the shape dependence of human listeners' sensitivity to interaural temporal differences (ITDs) in the ongoing envelope, were used to measure the lateralization produced by fixed interaural disparities. Consistent with the threshold ITD data, a steep attack and a non-zero pause facilitate strong ITD-based lateralization. In contrast, those conditions resulted in the smallest interaural level-based lateralization.

Temporal weighting functions for interaural time and level differences. IV. Effects of carrier frequency

Temporal variation in listeners' sensitivity to interaural time and level differences (ITD and ILD, respectively) was measured for sounds of different carrier frequency using the temporal weighting function (TWF) paradigm [Stecker and Hafter (2002) J. Acoust. Soc. Am. 112,1046-1057]. Listeners made lateralization judgments following brief trains of filtered impulses (Gabor clicks) presented over headphones with overall ITD and/or ILD ranging from ±500 μs ITD and/or ±5 dB ILD across trials. Individual clicks within each train varied by an additional ±100 μs ITD or ±2 dB ILD to allow TWF calculation by multiple regression. In separate conditions, TWFs were measured for carrier frequencies of 1, 2, 4, and 8 kHz. Consistent with past studies, TWFs demonstrated high weight on the first click for stimuli with short interclick interval (ICI = 2 ms), but flatter weighting for longer ICI (5-10 ms). Some conditions additionally demonstrated greater weight for clicks near the offset than near the middle of the train. Results support a primary role of the auditory periphery in emphasizing onset and offset cues in rapidly modulated low-frequency sounds. For slower modulations, sensitivity to ongoing high-frequency ILD and low-frequency ITD cues appears subject to recency effects consistent with the effects of leaky temporal integration of binaural information.

The natural history of sound localization in mammals--a story of neuronal inhibition

Our concepts of sound localization in the vertebrate brain are widely based on the general assumption that both the ability to detect air-borne sounds and the neuronal processing are homologous in archosaurs (present day crocodiles and birds) and mammals. Yet studies repeatedly report conflicting results on the neuronal circuits and mechanisms, in particular the role of inhibition, as well as the coding strategies between avian and mammalian model systems. Here we argue that mammalian and avian phylogeny of spatial hearing is characterized by a convergent evolution of hearing air-borne sounds rather than by homology. In particular, the different evolutionary origins of tympanic ears and the different availability of binaural cues in early mammals and archosaurs imposed distinct constraints on the respective binaural processing mechanisms. The role of synaptic inhibition in generating binaural spatial sensitivity in mammals is highlighted, as it reveals a unifying principle of mammalian circuit design for encoding sound position. Together, we combine evolutionary, anatomical and physiological arguments for making a clear distinction between mammalian processing mechanisms and coding strategies and those of archosaurs. We emphasize that a consideration of the convergent nature of neuronal mechanisms will significantly increase the explanatory power of studies of spatial processing in both mammals and birds.