Interaction in the perceptual processing of interaural time and level differences

Binaural spatial adaptation as a mechanism for asymmetric trading of interaural time and level differences

A classic paradigm used to quantify the perceptual weighting of binaural spatial cues requires a listener to adjust the value of one cue, while the complementary cue is held constant. Adjustments are made until the auditory percept appears centered in the head, and the values of both cues are recorded as a trading relation (TR), most commonly in μs interaural time difference per dB interaural level difference. Interestingly, existing literature has shown that TRs differ according to the cue being adjusted. The current study investigated whether cue-specific adaptation, which might arise due to the continuous, alternating presentation of signals during adjustment tasks, could account for this poorly understood phenomenon. Three experiments measured TRs via adjustment and via lateralization of single targets in virtual reality (VR). Targets were 500 Hz pure tones preceded by silence or by adapting trains that held one of the cues constant. VR removed visual anchors and provided an intuitive response technique during lateralization. The pattern of results suggests that adaptation can account for cue-dependent TRs. In addition, VR seems to be a viable tool for psychophysical tasks.

Cortical mechanisms of spatial hearing

Humans and other animals use spatial hearing to rapidly localize events in the environment. However, neural encoding of sound location is a complex process involving the computation and integration of multiple spatial cues that are not represented directly in the sensory organ (the cochlea). Our understanding of these mechanisms has increased enormously in the past few years. Current research is focused on the contribution of animal models for understanding human spatial audition, the effects of behavioural demands on neural sound location encoding, the emergence of a cue-independent location representation in the auditory cortex, and the relationship between single-source and concurrent location encoding in complex auditory scenes. Furthermore, computational modelling seeks to unravel how neural representations of sound source locations are derived from the complex binaural waveforms of real-life sounds. In this article, we review and integrate the latest insights from neurophysiological, neuroimaging and computational modelling studies of mammalian spatial hearing. We propose that the cortical representation of sound location emerges from recurrent processing taking place in a dynamic, adaptive network of early (primary) and higher-order (posterior-dorsal and dorsolateral prefrontal) auditory regions. This cortical network accommodates changing behavioural requirements and is especially relevant for processing the location of real-life, complex sounds and complex auditory scenes.

Extraction of Inter-Aural Time Differences Using a Spiking Neuron Network Model of the Medial Superior Olive

The mammalian auditory system is able to extract temporal and spectral features from sound signals at the two ears. One important cue for localization of low-frequency sound sources in the horizontal plane are inter-aural time differences (ITDs) which are first analyzed in the medial superior olive (MSO) in the brainstem. Neural recordings of ITD tuning curves at various stages along the auditory pathway suggest that ITDs in the mammalian brainstem are not represented in form of a Jeffress-type place code. An alternative is the hemispheric opponent-channel code, according to which ITDs are encoded as the difference in the responses of the MSO nuclei in the two hemispheres. In this study, we present a physiologically-plausible, spiking neuron network model of the mammalian MSO circuit and apply two different methods of extracting ITDs from arbitrary sound signals. The network model is driven by a functional model of the auditory periphery and physiological models of the cochlear nucleus and the MSO. Using a linear opponent-channel decoder, we show that the network is able to detect changes in ITD with a precision down to 10 μs and that the sensitivity of the decoder depends on the slope of the ITD-rate functions. A second approach uses an artificial neuronal network to predict ITDs directly from the spiking output of the MSO and ANF model. Using this predictor, we show that the MSO-network is able to reliably encode static and time-dependent ITDs over a large frequency range, also for complex signals like speech.

Adaptation of Central Pitch-Specific Mechanisms

Three previous psychophysical studies have demonstrated that interaural time difference (ITD) coding mechanisms can undergo frequency-specific, selective adaptation. We sought to determine whether this phenomenon extends to the pitch domain, by employing the same psychophysical paradigm as one used previously, but with harmonic tone complexes lacking energy at the fundamental frequency. Ten normal listeners participated in experiment 1. Psychometric functions for ITDs were obtained for harmonic tone complexes with fundamental frequencies of 110 Hz and 185 Hz, before and after selective adaptation with complexes of the same fundamental frequencies lateralised to opposite sides. In experiment 1, each subject was tested twice. On separate days, subjects were tested with 110 Hz and 185 Hz stimuli that were either partially resolvable complexes or unresolvable ones. Both partially resolved and unresolved stimuli supported adaptation, and at both fundamental frequencies. In experiment 2, which employed nine listeners, the adaptor tone complexes were presented in conjunction with a diotic noise background designed to mask difference tones generated by the adaptor stimuli. The use of the masker had little effect on the mean strength of the adaptation effected by the unresolved adaptor stimuli, and only slightly weakened the adaptation effect found with the partially resolved adaptor stimuli. Taken together, these data constitute the first demonstration of selective adaptation exerted on a central mechanism in the pitch domain.

Physiological Evidence for a Midline Spatial Channel in Human Auditory Cortex

Studies with humans and other mammals have provided support for a two-channel representation of horizontal (“azimuthal”) space in the auditory system. In this representation, location-sensitive neurons contribute activity to one of two broadly tuned channels whose responses are compared to derive an estimate of sound-source location. One channel is maximally responsive to sounds towards the left and the other to sounds towards the right. However, recent psychophysical studies of humans, and physiological studies of other mammals, point to the presence of an additional channel, maximally responsive to the midline. In this study, we used electroencephalography to seek physiological evidence for such a midline channel in humans. We measured neural responses to probe stimuli presented from straight ahead (0 °) or towards the right (+30 ° or +90 °). Probes were preceded by adapter stimuli to temporarily suppress channel activity. Adapters came from 0 ° or alternated between left and right (−30 ° and +30 ° or −90 ° and +90 °). For the +90 ° probe, to which the right-tuned channel would respond most strongly, both accounts predict greatest adaptation when the adapters are at ±90 °. For the 0 ° probe, the two-channel account predicts greatest adaptation from the ±90 ° adapters, while the three-channel account predicts greatest adaptation when the adapters are at 0 ° because these adapters stimulate the midline-tuned channel which responds most strongly to the 0 ° probe. The results were consistent with the three-channel account. In addition, a computational implementation of the three-channel account fitted the probe response sizes well, explaining 93 % of the variance about the mean, whereas a two-channel implementation produced a poor fit and explained only 61 % of the variance.

Opponent Coding of Sound Location (Azimuth) in Planum Temporale is Robust to Sound-Level Variations

Coding of sound location in auditory cortex (AC) is only partially understood. Recent electrophysiological research suggests that neurons in mammalian auditory cortex are characterized by broad spatial tuning and a preference for the contralateral hemifield, that is, a nonuniform sampling of sound azimuth. Additionally, spatial selectivity decreases with increasing sound intensity. To accommodate these findings, it has been proposed that sound location is encoded by the integrated activity of neuronal populations with opposite hemifield tuning (“opponent channel model”). In this study, we investigated the validity of such a model in human AC with functional magnetic resonance imaging (fMRI) and a phase-encoding paradigm employing binaural stimuli recorded individually for each participant. In all subjects, we observed preferential fMRI responses to contralateral azimuth positions. Additionally, in most AC locations, spatial tuning was broad and not level invariant. We derived an opponent channel model of the fMRI responses by subtracting the activity of contralaterally tuned regions in bilateral planum temporale. This resulted in accurate decoding of sound azimuth location, which was unaffected by changes in sound level. Our data thus support opponent channel coding as a neural mechanism for representing acoustic azimuth in human AC.

Are interaural time and level differences represented by independent or integrated codes in the human auditory cortex?

Sound localization is important for orienting and focusing attention and for segregating sounds from different sources in the environment. In humans, horizontal sound localization mainly relies on interaural differences in sound arrival time and sound level. Despite their perceptual importance, the neural processing of interaural time and level differences (ITDs and ILDs) remains poorly understood. Animal studies suggest that, in the brainstem, ITDs and ILDs are processed independently by different specialized circuits. The aim of the current study was to investigate whether, at higher processing levels, they remain independent or are integrated into a common code of sound laterality. For that, we measured late auditory cortical potentials in response to changes in sound lateralization elicited by perceptually matched changes in ITD and/or ILD. The responses to the ITD and ILD changes exhibited significant morphological differences. At the same time, however, they originated from overlapping areas of the cortex and showed clear evidence for functional coupling. These results suggest that the auditory cortex contains an integrated code of sound laterality, but also retains independent information about ITD and ILD cues. This cue-related information might be used to assess how consistent the cues are, and thus, how likely they would have arisen from the same source.

Evidence for opponent process analysis of sound source location in humans

Research with barn owls suggested that sound source location is represented topographically in the brain by an array of neurons each tuned to a narrow range of locations. However, research with small-headed mammals has offered an alternative view in which location is represented by the balance of activity in two opponent channels broadly tuned to the left and right auditory space. Both channels may be present in each auditory cortex, although the channel representing contralateral space may be dominant. Recent studies have suggested that opponent channel coding of space may also apply in humans, although these studies have used a restricted set of spatial cues or probed a restricted set of spatial locations, and there have been contradictory reports as to the relative dominance of the ipsilateral and contralateral channels in each cortex. The current study used electroencephalography (EEG) in conjunction with sound field stimulus presentation to address these issues and to inform the development of an explicit computational model of human sound source localization. Neural responses were compatible with the opponent channel account of sound source localization and with contralateral channel dominance in the left, but not the right, auditory cortex. A computational opponent channel model reproduced every important aspect of the EEG data and allowed inferences about the width of tuning in the spatial channels. Moreover, the model predicted the oft-reported decrease in spatial acuity measured psychophysically with increasing reference azimuth. Predictions of spatial acuity closely matched those measured psychophysically by previous authors.

Adaptive coding is constrained to midline locations in a spatial listening task

Many neurons adapt their spike output to accommodate the prevailing sensory environment. Although such adaptation is thought to improve coding of relevant stimulus features, the relationship between adaptation at the neural and behavioral levels remains to be established. Here we describe improved discrimination performance for an auditory spatial cue (interaural time differences, ITDs) following adaptation to stimulus statistics. Physiological recordings in the midbrain of anesthetized guinea pigs and measurement of discrimination performance in humans both demonstrate improved coding of the most prevalent ITDs in a distribution, but with highest accuracy maintained for ITDs corresponding to frontal locations, suggesting the existence of a fovea for auditory space. A biologically plausible model accounting for the physiological data suggests that neural tuning is stabilized by inhibition to maintain high discriminability for frontal locations. The data support the notion that adaptive coding in the midbrain is a key element of behaviorally efficient sound localization in dynamic acoustic environments.

The three-channel model of sound localization mechanisms: interaural level differences

The current understanding of mammalian sound localization is that azimuthal (horizontal) position assignments are dependent upon the relative activation of two populations of broadly-tuned hemifield neurons with overlapping medial borders. Recent psychophysical work has provided evidence for a third channel of low-frequency interaural time difference (ITD)-sensitive neurons tuned to the azimuthal midline. However, the neurophysiological data on free-field azimuth receptive fields, especially of cortical neurons, has primarily studied high-frequency cells whose receptive fields are more likely to have been shaped by interaural level differences (ILDs) than ITDs. In four experiments, a selective adaptation paradigm was used to probe for the existence of a midline channel in the domain of ILDs. If no midline channel exists, symmetrical adaptation of the lateral channels should not result in a shift in the perceived intracranial location of subsequent test tones away from the adaptors because the relative activation of the two channels will remain unchanged. Instead, results indicate a shift in perceived test tone location away from the adaptors, which supports the existence of a midline channel in the domain of ILDs. Interestingly, this shift occurs not only at high frequencies, traditionally associated with ILDs in natural settings, but at low frequencies as well.

Auditory Spatial Processing in the Human Cortex

The auditory system codes spatial locations in a way that deviates from the spatial representations found in other modalities. This difference is especially striking in the cortex, where neurons form topographical maps of visual and tactile space but where auditory space is represented through a population rate code. In this hemifield code, sound source location is represented in the activity of two widely tuned opponent populations, one tuned to the right and the other to the left side of auditory space. Scientists are only beginning to uncover how this coding strategy adapts to various spatial processing demands. This review presents the current understanding of auditory spatial processing in the cortex. To this end, the authors consider how various implementations of the hemifield code may exist within the auditory cortex and how these may be modulated by the stimulation and task context. As a result, a coherent set of neural strategies for auditory spatial processing emerges.

A midline azimuthal channel in human spatial hearing

Neurophysiological and psychophysical evidence has driven the formulation of a hemifield model of mammalian sound localization in which the perceived location of a stimulus is based on the relative activity of two hemifield-tuned azimuthal channels that are broadly responsive to contralateral auditory space and have overlapping medial borders. However, neurophysiological work in mammals has consistently found neurons selective for sound sources at the midline, which may indicate the existence of a third, 'midline', perceptual channel. In three experiments, the existence of three (left, right, midline) perceptual channels for azimuth in man was examined using auditory selective adaptation paradigms. If no midline channel exists, exposure to highly lateralized, symmetrical adaptor frequencies should not result in a shift in the perceived intracranial location of subsequent test tones away from the adaptors because the relative activation of the two hemifield channels will remain the same. Rather, our results indicate a shift in perceived test tones towards the azimuthal midline. This result can best be explained by a perceptual/neural channel tuned to sounds located along the midline. The present study gives the first psychophysical evidence of a midline channel serving human auditory localization, adding to the earlier evidence on the same point from animal neurophysiological studies.

The neural code for interaural time difference in human auditory cortex

A magnetoencephalography study was conducted to reveal the neural code of interaural time difference (ITD) in the human cortex. Widely used crosscorrelator models predict that the code consists of narrow receptive fields distributed to all ITDs. The present findings are, however, more in line with a neural code formed by two opponent neural populations: one tuned to the left and the other to the right hemifield. The results are consistent with models of ITD extraction in the auditory brainstem of small mammals and, therefore, suggest that similar computational principles underlie human sound source localization.

Context effects in the discriminability of spatial cues

In order to investigate whether performance in an auditory spatial discrimination task depends on the prevailing listening conditions, we tested the ability of human listeners to discriminate target sounds with and without presentation of a preceding sound. Target sounds were either lateralized by means of interaural time differences (ITDs) of +400, 0, or -400 micros or interaural level differences (ILDs) with the same subjective intracranial locations. The preceding sound was always lateralized by means of ITD. This allowed for testing whether the effects of a preceding sound were location- or cue-specific. Preceding sounds and target sounds were randomly paired across trials. Listeners had to discriminate whether they perceived the target sounds as coming from the same or different intracranial locations. Finally, stimuli were selected so that, without any preceding sound, ITD and ILD cues were equally discriminable at all target lateralizations. Stimuli were 800 Hz-wide, 400-ms duration bands of noise centered at 500 Hz, presented over headphones. The duration of the preceding sound was randomly selected from a uniform distribution spanning from 1s to 2s. Results show that discriminability of both binaural cues was improved for midline target positions when preceding sound and targets were co-located, whereas it was impaired when preceding sound and targets came from different positions. No effect of the preceding sound was found for left or right target positions. These results are compatible with a purely bottom-up mechanism based on adaptive coding of ITD around the midline that may be combined with top-down mechanisms to increase localization accuracy in realistic listening conditions.

The effects of lateralized adaptors on lateral position judgements of tones within and across frequency channels

Two experiments examined the effect of highly lateralized adaptor tone pulses on the perceived intracranial location of subsequent test tones. In Experiment 1, adaptor tones of each of two frequencies, highly lateralized to opposite sides by a quarter-period interaural time difference (ITD), were found to shift the perceived intracranial location of test tones of each adaptor frequency away from the side of the adaptor. The shift in perceived location was seen for all test tone ITDs with the same sign as the adaptor tone, and sometimes extended to include test tones with small ITDs favoring the opposite ear. The generality of the effect across test tone ITDs of the same sign as the adaptor suggests that the human auditory lateralization system is built of two (left, right) hemifield-tuned azimuthal channels, and that perceived lateral location depends on the relative outputs of those two channels. In Experiment 2, the perceived location of test tones lateralized by ITD was studied in the same listeners at each of the same two frequencies, but after selective adaptation with tone pulses of only one frequency and laterality. The perceived lateral position of test tones with the same frequency as that of the adaptor underwent the same changes as seen in Experiment 1. The perceived lateral position of test tones of the nonadapted frequency usually shifted weakly in the opposite direction, i.e., in the direction expected if the second adaptor from Experiment 1 had actually been present. These data have implications both for the processes mediating selective adaptation using contingent stimuli, and for the azimuthal tuning of auditory spatial channels in man.