When and where of auditory spatial processing in cortex: a novel approach using electrotomography

Examining the lateralization of electrophysiological correlates of auditory awareness

The neurological basis for perceptual awareness remains unclear, and theories disagree as to whether sensory cortices per se generate awareness. Critically, neural activity in the sensory cortices is only a neural correlate of consciousness (NCC) if it closely matches the contents of perceptual awareness. Research in vision and touch suggest that contralateral activity in sensory cortices is an NCC. Similarly, research in hearing with two sound sources (left and right) presented over headphones also suggests that a candidate NCC called the auditory awareness negativity (AAN) matches perceived location of sound. The current study used 13 different sound sources presented over loudspeakers for natural localization cues and measured event-related potentials to a threshold stimulus in a sound localization task. Preregistered Bayesian mixed models provided moderate evidence against an overall AAN and very strong evidence against its lateralization. Because of issues regarding data quantity and quality, exploratory analyses with aggregated data from multiple loudspeakers were conducted. Results provided moderate evidence for an overall AAN and strong evidence against its lateralization. Nonetheless, the interpretations of these results remain inconclusive. Therefore, future research should reduce the number of conditions and/or test over several sessions to procure a sufficient amount of data. Taken at face value, the results may suggest issues with AAN as an NCC of auditory awareness, as it does not laterally map onto experiences in a free-field auditory environment, in contrast to the NCCs of vision and touch.

Electrophysiological correlation of auditory selective spatial attention in the "cocktail party" situation

The auditory system can selectively attend to the target source in complex environments, the phenomenon known as the "cocktail party" effect. However, the spatiotemporal dynamics of electrophysiological activity associated with auditory selective spatial attention (ASSA) remain largely unexplored. In this study, single-source and multiple-source paradigms were designed to simulate different auditory environments, and microstate analysis was introduced to reveal the electrophysiological correlates of ASSA. Furthermore, cortical source analysis was employed to reveal the neural activity regions of these microstates. The results showed that five microstates could explain the spatiotemporal dynamics of ASSA, ranging from MS1 to MS5. Notably, MS2 and MS3 showed significantly lower partial properties in multiple-source situations than in single-source situations, whereas MS4 had shorter durations and MS5 longer durations in multiple-source situations than in single-source situations. MS1 had insignificant differences between the two situations. Cortical source analysis showed that the activation regions of these microstates initially transferred from the right temporal cortex to the temporal-parietal cortex, and subsequently to the dorsofrontal cortex. Moreover, the neural activity of the single-source situations was greater than that of the multiple-source situations in MS2 and MS3, correlating with the N1 and P2 components, with the greatest differences observed in the superior temporal gyrus and inferior parietal lobule. These findings suggest that these specific microstates and their associated activation regions may serve as promising substrates for decoding ASSA in complex environments.

Mismatch negativity between discriminating and undiscriminating participants on the front-back sound localization

Sound localization in the front-back dimension is reported to be challenging, with individual differences. We investigated whether auditory discrimination processing in the brain differs based on front-back sound localization ability. This study conducted an auditory oddball task using speakers in front of and behind the participants. We used event-related brain potentials to examine the deviance detection process between groups that could and could not discriminate front-back sound localization. The results indicated that mismatch negativity (MMN) occurred during the deviance detection process, and P2 amplitude differed between standard and deviant locations in both groups. However, the latency of MMN was shorter in the group that could discriminate front-back sounds than in the group that could not. Additionally, N1 amplitude increased for deviant locations compared to standard ones only in the discriminating group. In conclusion, the sensory memories matching process based on traces of previously presented stimuli (MMN, P2) occurred regardless of discrimination ability. However, the response to changes in the physical properties of sounds (MMN latency, N1 amplitude) differed depending on the ability to discriminate front-back sounds. Our findings suggest that the brain may have different processing strategies for the two directions even without subjective recognition of the front-back direction of incoming sounds.

Comparing auditory distance perception in real and virtual environments and the role of the loudness cue: A study based on event-related potentials

The perception of the distance to a sound source is relevant in many everyday situations, not only in real spaces, but also in virtual reality (VR) environments. Where real rooms often reach their limits, VR offers far-reaching possibilities to simulate a wide range of acoustic scenarios. However, in virtual room acoustics a plausible reproduction of distance-related cues can be challenging. In the present study, we compared the detection of changes of the distance to a sound source and its neurocognitive correlates in a real and a virtual reverberant environment, using an active auditory oddball paradigm and EEG measures. The main goal was to test whether the experiments in the virtual and real environments produced equivalent behavioral and EEG results. Three loudspeakers were placed at ego-centric distances of 2 m (near), 4 m (center), and 8 m (far) in front of the participants (N = 20), each 66 cm below their ear level. Sequences of 500 ms noise stimuli were presented either from the center position (standards, 80 % of trials) or from the near or far position (targets, 10 % each). The participants had to indicate a target position via a joystick response ("near" or "far"). Sounds were emitted either by real loudspeakers in the real environment or rendered and played back for the corresponding positions via headphones in the virtual environment. In addition, within both environments, loudness of the auditory stimuli was either unaltered (natural loudness) or the loudness cue was manipulated, so that all three loudspeakers were perceived equally loud at the listener's position (matched loudness). The EEG analysis focused on the mismatch negativity (MMN), P3a, and P3b as correlates of deviance detection, attentional orientation, and context-updating/stimulus evaluation, respectively. Overall, behavioral data showed that detection of the target positions was reduced within the virtual environment, and especially when loudness was matched. Except for slight latency shifts in the virtual environment, EEG analysis indicated comparable patterns within both environments and independent of loudness settings. Thus, while the neurocognitive processing of changes in distance appears to be similar in virtual and real spaces, a proper representation of loudness appears to be crucial to achieve a good task performance in virtual acoustic environments.

The scalp time-varying network of auditory spatial attention in "cocktail-party" situations

Sound source localization in "cocktail-party" situations is a remarkable ability of the human auditory system. However, the neural mechanisms underlying auditory spatial attention are still largely unknown. In this study, the "cocktail-party" situations are simulated through multiple sound sources and presented through head-related transfer functions and headphones. Furthermore, the scalp time-varying network of auditory spatial attention is constructed using the high-temporal resolution electroencephalogram, and its network properties are measured quantitatively using graph theory analysis. The results show that the time-varying network of auditory spatial attention in "cocktail-party" situations is more complex and partially different than in simple acoustic situations, especially in the early- and middle-latency periods. The network coupling strength increases continuously over time, and the network hub shifts from the posterior temporal lobe to the parietal lobe and then to the frontal lobe region. In addition, the right hemisphere has a stronger network strength for processing auditory spatial information in "cocktail-party" situations, i.e., the right hemisphere has higher clustering levels, higher transmission efficiency, and more node degrees during the early- and middle-latency periods, while this phenomenon disappears and appears symmetrically during the late-latency period. These findings reveal different network patterns and properties of auditory spatial attention in "cocktail-party" situations during different periods and demonstrate the dominance of the right hemisphere in the dynamic processing of auditory spatial information.

Cortical temporal integration can account for limits of temporal perception: investigations in the binaural system

The auditory system has exquisite temporal coding in the periphery which is transformed into a rate-based code in central auditory structures, like auditory cortex. However, the cortex is still able to synchronize, albeit at lower modulation rates, to acoustic fluctuations. The perceptual significance of this cortical synchronization is unknown. We estimated physiological synchronization limits of cortex (in humans with electroencephalography) and brainstem neurons (in chinchillas) to dynamic binaural cues using a novel system-identification technique, along with parallel perceptual measurements. We find that cortex can synchronize to dynamic binaural cues up to approximately 10 Hz, which aligns well with our measured limits of perceiving dynamic spatial information and utilizing dynamic binaural cues for spatial unmasking, i.e. measures of binaural sluggishness. We also find that the tracking limit for frequency modulation (FM) is similar to the limit for spatial tracking, demonstrating that this sluggish tracking is a more general perceptual limit that can be accounted for by cortical temporal integration limits.

Effects of Temporal Characteristics on Pilots Perceiving Audiovisual Warning Signals Under Different Perceptual Loads

Our research aimed to investigate the effectiveness of auditory, visual, and audiovisual warning signals for capturing the attention of the pilot, and how stimulus onset asynchronies (SOA) in audiovisual stimuli affect pilots perceiving the bimodal warning signals under different perceptual load conditions. In experiment 1 of the low perceptual load condition, participants discriminated the location (right vs. left) of visual targets preceded by five different types of warning signals. In experiment 2 of high perceptual load, participants completed the location task identical to a low load condition and a digit detection task in a rapid serial visual presentation (RSVP) stream. The main effect of warning signals in two experiments showed that visual and auditory cues presented simultaneously (AV) could effectively and efficiently arouse the attention of the pilots in high and low load conditions. Specifically, auditory (A), AV, and visual preceding auditory stimulus by 100 ms (VA100) increased the spatial orientation to a valid position in low load conditions. With the increase in visual perceptual load, auditory preceding the visual stimulus by 100 ms (AV100) and A warning signals had stronger spatial orientation. The results are expected to theoretically support the optimization design of the cockpit display interface, contributing to immediate flight crew awareness.

A structured ICA-based process for removing auditory evoked potentials

Transcranial magnetic stimulation (TMS)-evoked potentials (TEPs), recorded using electroencephalography (EEG), reflect a combination of TMS-induced cortical activity and multi-sensory responses to TMS. The auditory evoked potential (AEP) is a high-amplitude sensory potential-evoked by the "click" sound produced by every TMS pulse-that can dominate the TEP and obscure observation of other neural components. The AEP is peripherally evoked and therefore should not be stimulation site specific. We address the problem of disentangling the peripherally evoked AEP of the TEP from components evoked by cortical stimulation and ask whether removal of AEP enables more accurate isolation of TEP. We hypothesized that isolation of the AEP using Independent Components Analysis (ICA) would reveal features that are stimulation site specific and unique individual features. In order to improve the effectiveness of ICA for removal of AEP from the TEP, and thus more clearly separate the transcranial-evoked and non-specific TMS-modulated potentials, we merged sham and active TMS datasets representing multiple stimulation conditions, removed the resulting AEP component, and evaluated performance across different sham protocols and clinical populations using reduction in Global and Local Mean Field Power (GMFP/LMFP) and cosine similarity analysis. We show that removing AEPs significantly reduced GMFP and LMFP in the post-stimulation TEP (14 to 400 ms), driven by time windows consistent with the N100 and P200 temporal characteristics of AEPs. Cosine similarity analysis supports that removing AEPs reduces TEP similarity between subjects and reduces TEP similarity between stimulation conditions. Similarity is reduced most in a mid-latency window consistent with the N100 time-course, but nevertheless remains high in this time window. Residual TEP in this window has a time-course and topography unique from AEPs, which follow-up exploratory analyses suggest could be a modulation in the alpha band that is not stimulation site specific but is unique to individual subject. We show, using two datasets and two implementations of sham, evidence in cortical topography, TEP time-course, GMFP/LMFP and cosine similarity analyses that this procedure is effective and conservative in removing the AEP from TEP, and may thus better isolate TMS-evoked activity. We show TEP remaining in early, mid and late latencies. The early response is site and subject specific. Later response may be consistent with TMS-modulated alpha activity that is not site specific but is unique to the individual. TEP remaining after removal of AEP is unique and can provide insight into TMS-evoked potentials and other modulated oscillatory dynamics.

Preserved sensory processing but hampered conflict detection when stimulus input is task-irrelevant

Conflict detection in sensory input is central to adaptive human behavior. Perhaps unsurprisingly, past research has shown that conflict may even be detected in the absence of conflict awareness, suggesting that conflict detection is an automatic process that does not require attention. To test the possibility of conflict processing in the absence of attention, we manipulated task relevance and response overlap of potentially conflicting stimulus features across six behavioral tasks. Multivariate analyses on human electroencephalographic data revealed neural signatures of conflict only when at least one feature of a conflicting stimulus was attended, regardless of whether that feature was part of the conflict, or overlaps with the response. In contrast, neural signatures of basic sensory processes were present even when a stimulus was completely unattended. These data reveal an attentional bottleneck at the level of objects, suggesting that object-based attention is a prerequisite for cognitive control operations involved in conflict detection.

Focusing your attention on one thing can leave you surprisingly unaware of what goes on around you. A classic experiment known as ‘the invisible gorilla’ highlights this phenomenon. Volunteers were asked to watch a clip featuring basketball players, and count how often those wearing white shirts passed the ball: around half of participants failed to spot that someone wearing a gorilla costume wandered into the game and spent nine seconds on screen.

Yet, things that you are not focusing on can sometimes grab your attention anyway. Take for example, the ‘cocktail party effect’, the ability to hear your name among the murmur of a crowded room. So why can we react to our own names, but fail to spot the gorilla? To help answer this question, Nuiten et al. examined how paying attention affects the way the brain processes input.

Healthy volunteers were asked to perform various tasks while the words ‘left’ or ‘right’ played through speakers. The content of the word was sometimes consistent with its location (‘left’ being played on the left speaker), and sometimes opposite (‘left’ being played on the right speaker). Processing either the content or the location of the word is relatively simple for the brain; however detecting a discrepancy between these two properties is challenging, requiring the information to be processed in a brain region that monitors conflict in sensory input.

To manipulate whether the volunteers needed to pay attention to the words, Nuiten et al. made their content or location either relevant or irrelevant for a task. By analyzing brain activity and task performance, they were able to study the effects of attention on how the word properties were processed.

The results showed that the volunteers’ brains were capable of dealing with basic information, such as location or content, even when their attention was directed elsewhere. But discrepancies between content and location could only be detected when the volunteers were focusing on the words, or when their content or location was directly relevant to the task.

The findings by Nuiten et al. suggest that while performing a difficult task, our brains continue to react to basic input but often fail to process more complex information. This, in turn, has implications for a range of human activities such as driving. New technology could potentially help to counteract this phenomenon, aiming to direct attention towards complex information that might otherwise be missed.

What and where in the auditory systems of sighted and early blind individuals: Evidence from representational similarity analysis

Separated ventral and dorsal streams in auditory system have been proposed to process sound identification and localization respectively. Despite the popularity of the dual-pathway model, it remains controversial how much independence two neural pathways enjoy and whether visual experiences can influence the distinct cortical organizational scheme. In this study, representational similarity analysis (RSA) was used to explore the functional roles of distinct cortical regions that lay within either the ventral or dorsal auditory streams of sighted and early blind (EB) participants. We found functionally segregated auditory networks in both sighted and EB groups where anterior superior temporal gyrus (aSTG) and inferior frontal junction (IFJ) were more related to the sound identification, while posterior superior temporal gyrus (pSTG) and inferior parietal lobe (IPL) preferred the sound localization. The findings indicated visual experiences may not have an influence on this functional dissociation and the cortex of the human brain may be organized as task-specific and modality-independent strategies. Meanwhile, partial overlap of spatial and non-spatial auditory information processing was observed, illustrating the existence of interaction between the two auditory streams. Furthermore, we investigated the effect of visual experiences on the neural bases of auditory perception and observed the cortical reorganization in EB participants in whom middle occipital gyrus was recruited to process auditory information. Our findings examined the distinct cortical networks that abstractly encoded sound identification and localization, and confirmed the existence of interaction from the multivariate perspective. Furthermore, the results suggested visual experience might not impact the functional specialization of auditory regions.

Revealing the differences of the representations of sounds from different directions in the human brain using functional connectivity

Many studies have focused on the processing mechanism of sound directions in the human brain, however, as far as we know, it remains unclear whether the representations of sounds from different directions are different. In the present study, 28 subjects were scanned while listening to sounds from different directions. We used the whole-brain functional connectivity (FC) analysis to explore which brain regions had significant changes. Our results revealed that sounds from different directions affected the FC in the widely distributed regions. Importantly, all regions showed significant differences in FC between the central and eccentric directions, while few regions showed a difference between the left and right directions. These findings revealed the differences in the representations of sounds from different directions.

Mapping the human middle longitudinal fasciculus through a focused anatomo-imaging study: shifting the paradigm of its segmentation and connectivity pattern

Τhe middle longitudinal fasciculus (MdLF) was initially identified in humans as a discrete subcortical pathway connecting the superior temporal gyrus (STG) to the angular gyrus (AG). Further anatomo-imaging studies, however, proposed more sophisticated but conflicting connectivity patterns and have created a vague perception on its functional anatomy. Our aim was, therefore, to investigate the ambiguous structural architecture of this tract through focused cadaveric dissections augmented by a tailored DTI protocol in healthy participants from the Human Connectome dataset. Three segments and connectivity patterns were consistently recorded: the MdLF-I, connecting the dorsolateral Temporal Pole (TP) and STG to the Superior Parietal Lobule/Precuneus, through the Heschl's gyrus; the MdLF-II, connecting the dorsolateral TP and the STG with the Parieto-occipital area through the posterior transverse gyri and the MdLF-III connecting the most anterior part of the TP to the posterior border of the occipital lobe through the AG. The lack of an established termination pattern to the AG and the fact that no significant leftward asymmetry is disclosed tend to shift the paradigm away from language function. Conversely, the theory of "where" and "what" auditory pathways, the essential relationship of the MdLF with the auditory cortex and the functional role of the cortical areas implicated in its connectivity tend to shift the paradigm towards auditory function. Allegedly, the MdLF-I and MdLF-II segments could underpin the perception of auditory representations; whereas, the MdLF-III could potentially subserve the integration of auditory and visual information.

Where is the cocktail party? Decoding locations of attended and unattended moving sound sources using EEG

Recently, we showed that in a simple acoustic scene with one sound source, auditory cortex tracks the time-varying location of a continuously moving sound. Specifically, we found that both the delta phase and alpha power of the electroencephalogram (EEG) can be used to reconstruct the sound source azimuth. However, in natural settings, we are often presented with a mixture of multiple competing sounds and so we must focus our attention on the relevant source in order to segregate it from the competing sources e.g. 'cocktail party effect'. While many studies have examined this phenomenon in the context of sound envelope tracking by the cortex, it is unclear how we process and utilize spatial information in complex acoustic scenes with multiple sound sources. To test this, we created an experiment where subjects listened to two concurrent sound stimuli that were moving within the horizontal plane over headphones while we recorded their EEG. Participants were tasked with paying attention to one of the two presented stimuli. The data were analyzed by deriving linear mappings, temporal response functions (TRF), between EEG data and attended as well unattended sound source trajectories. Next, we used these TRFs to reconstruct both trajectories from previously unseen EEG data. In a first experiment we used noise stimuli and included the task involved spatially localizing embedded targets. Then, in a second experiment, we employed speech stimuli and a non-spatial speech comprehension task. Results showed the trajectory of an attended sound source can be reliably reconstructed from both delta phase and alpha power of EEG even in the presence of distracting stimuli. Moreover, the reconstruction was robust to task and stimulus type. The cortical representation of the unattended source position was below detection level for the noise stimuli, but we observed weak tracking of the unattended source location for the speech stimuli by the delta phase of EEG. In addition, we demonstrated that the trajectory reconstruction method can in principle be used to decode selective attention on a single-trial basis, however, its performance was inferior to envelope-based decoders. These results suggest a possible dissociation of delta phase and alpha power of EEG in the context of sound trajectory tracking. Moreover, the demonstrated ability to localize and determine the attended speaker in complex acoustic environments is particularly relevant for cognitively controlled hearing devices.

Transcranial direct current stimulation of posterior temporal cortex modulates electrophysiological correlates of auditory selective spatial attention in posterior parietal cortex

Speech perception in "cocktail-party" situations, in which a sound source of interest has to be extracted out of multiple irrelevant sounds, poses a remarkable challenge to the human auditory system. Studies on structural and electrophysiological correlates of auditory selective spatial attention revealed critical roles of the posterior temporal cortex and the N2 event-related potential (ERP) component in the underlying processes. Here, we explored effects of transcranial direct current stimulation (tDCS) to posterior temporal cortex on neurophysiological correlates of auditory selective spatial attention, with a specific focus on the N2. In a single-blind, sham-controlled crossover design with baseline and follow-up measurements, monopolar anodal and cathodal tDCS was applied for 16 min to the right posterior superior temporal cortex. Two age groups of human subjects, a younger (n = 20; age 18-30 yrs) and an older group (n = 19; age 66-77 yrs), completed an auditory free-field multiple-speakers localization task while ERPs were recorded. The ERP data showed an offline effect of anodal, but not cathodal, tDCS immediately after DC offset for targets contralateral, but not ipsilateral, to the hemisphere of tDCS, without differences between groups. This effect mainly consisted in a substantial increase of the N2 amplitude by 0.9 μV (SE 0.4 μV; d = 0.40) compared with sham tDCS. At the same point in time, cortical source localization revealed a reduction of activity in ipsilateral (right) posterior parietal cortex. Also, localization error was improved after anodal, but not cathodal, tDCS. Given that both the N2 and the posterior parietal cortex are involved in processes of auditory selective spatial attention, these results suggest that anodal tDCS specifically enhanced inhibitory attentional brain processes underlying the focusing onto a target sound source, possibly by improved suppression of irrelevant distracters.

Neural tracking of auditory motion is reflected by delta phase and alpha power of EEG

It is of increasing practical interest to be able to decode the spatial characteristics of an auditory scene from electrophysiological signals. However, the cortical representation of auditory space is not well characterized, and it is unclear how cortical activity reflects the time-varying location of a moving sound. Recently, we demonstrated that cortical response measures to discrete noise bursts can be decoded to determine their origin in space. Here we build on these findings to investigate the cortical representation of a continuously moving auditory stimulus using scalp recorded electroencephalography (EEG). In a first experiment, subjects listened to pink noise over headphones which was spectro-temporally modified to be perceived as randomly moving on a semi-circular trajectory in the horizontal plane. While subjects listened to the stimuli, we recorded their EEG using a 128-channel acquisition system. The data were analysed by 1) building a linear regression model (decoder) mapping the relationship between the stimulus location and a training set of EEG data, and 2) using the decoder to reconstruct an estimate of the time-varying sound source azimuth from the EEG data. The results showed that we can decode sound trajectory with a reconstruction accuracy significantly above chance level. Specifically, we found that the phase of delta (<2 Hz) and power of alpha (8-12 Hz) EEG track the dynamics of a moving auditory object. In a follow-up experiment, we replaced the noise with pulse train stimuli containing only interaural level and time differences (ILDs and ITDs respectively). This allowed us to investigate whether our trajectory decoding is sensitive to both acoustic cues. We found that the sound trajectory can be decoded for both ILD and ITD stimuli. Moreover, their neural signatures were similar and even allowed successful cross-cue classification. This supports the notion of integrated processing of ILD and ITD at the cortical level. These results are particularly relevant for application in devices such as cognitively controlled hearing aids and for the evaluation of virtual acoustic environments.

Supramodal effect of rightward prismatic adaptation on spatial representations within the ventral attentional system

Rightward prismatic adaptation (R-PA) was shown to alleviate not only visuo-spatial but also auditory symptoms in neglect. The neural mechanisms underlying the effect of R-PA have been previously investigated in visual tasks, demonstrating a shift of hemispheric dominance for visuo-spatial attention from the right to the left hemisphere both in normal subjects and in patients. We have investigated whether the same neural mechanisms underlie the supramodal effect of R-PA on auditory attention. Normal subjects underwent a brief session of R-PA, which was preceded and followed by an fMRI evaluation during which subjects detected targets within the left, central and right space in the auditory or visual modality. R-PA-related changes in activation patterns were found bilaterally in the inferior parietal lobule. In either modality, the representation of the left, central and right space increased in the left IPL, whereas the representation of the right space decreased in the right IPL. Thus, a brief exposure to R-PA modulated the representation of the auditory and visual space within the ventral attentional system. This shift in hemispheric dominance for auditory spatial attention offers a parsimonious explanation for the previously reported effects of R-PA on auditory symptoms in neglect.

Auditory-visual integration modulates location-specific repetition suppression of auditory responses

Space is a dimension shared by different modalities, but at what stage spatial encoding is affected by multisensory processes is unclear. Early studies observed attenuation of N1/P2 auditory evoked responses following repetition of sounds from the same location. Here, we asked whether this effect is modulated by audiovisual interactions. In two experiments, using a repetition-suppression paradigm, we presented pairs of tones in free field, where the test stimulus was a tone presented at a fixed lateral location. Experiment 1 established a neural index of auditory spatial sensitivity, by comparing the degree of attenuation of the response to test stimuli when they were preceded by an adapter sound at the same location versus 30° or 60° away. We found that the degree of attenuation at the P2 latency was inversely related to the spatial distance between the test stimulus and the adapter stimulus. In Experiment 2, the adapter stimulus was a tone presented from the same location or a more medial location than the test stimulus. The adapter stimulus was accompanied by a simultaneous flash displayed orthogonally from one of the two locations. Sound-flash incongruence reduced accuracy in a same-different location discrimination task (i.e., the ventriloquism effect) and reduced the location-specific repetition-suppression at the P2 latency. Importantly, this multisensory effect included topographic modulations, indicative of changes in the relative contribution of underlying sources across conditions. Our findings suggest that the auditory response at the P2 latency is affected by spatially selective brain activity, which is affected crossmodally by visual information.

Modulation of human auditory spatial scene analysis by transcranial direct current stimulation

Localizing and selectively attending to the source of a sound of interest in a complex auditory environment is an important capacity of the human auditory system. The underlying neural mechanisms have, however, still not been clarified in detail. This issue was addressed by using bilateral bipolar-balanced transcranial direct current stimulation (tDCS) in combination with a task demanding free-field sound localization in the presence of multiple sound sources, thus providing a realistic simulation of the so-called "cocktail-party" situation. With left-anode/right-cathode, but not with right-anode/left-cathode, montage of bilateral electrodes, tDCS over superior temporal gyrus, including planum temporale and auditory cortices, was found to improve the accuracy of target localization in left hemispace. No effects were found for tDCS over inferior parietal lobule or with off-target active stimulation over somatosensory-motor cortex that was used to control for non-specific effects. Also, the absolute error in localization remained unaffected by tDCS, thus suggesting that general response precision was not modulated by brain polarization. This finding can be explained in the framework of a model assuming that brain polarization modulated the suppression of irrelevant sound sources, thus resulting in more effective spatial separation of the target from the interfering sound in the complex auditory scene.

Testing the dual-pathway model for auditory processing in human cortex

Analogous to the visual system, auditory information has been proposed to be processed in two largely segregated streams: an anteroventral ("what") pathway mainly subserving sound identification and a posterodorsal ("where") stream mainly subserving sound localization. Despite the popularity of this assumption, the degree of separation of spatial and non-spatial auditory information processing in cortex is still under discussion. In the present study, a statistical approach was implemented to investigate potential behavioral dissociations for spatial and non-spatial auditory processing in stroke patients, and voxel-wise lesion analyses were used to uncover their neural correlates. The results generally provided support for anatomically and functionally segregated auditory networks. However, some degree of anatomo-functional overlap between "what" and "where" aspects of processing was found in the superior pars opercularis of right inferior frontal gyrus (Brodmann area 44), suggesting the potential existence of a shared target area of both auditory streams in this region. Moreover, beyond the typically defined posterodorsal stream (i.e., posterior superior temporal gyrus, inferior parietal lobule, and superior frontal sulcus), occipital lesions were found to be associated with sound localization deficits. These results, indicating anatomically and functionally complex cortical networks for spatial and non-spatial auditory processing, are roughly consistent with the dual-pathway model of auditory processing in its original form, but argue for the need to refine and extend this widely accepted hypothesis.

The mismatch negativity as a measure of auditory stream segregation in a simulated "cocktail-party" scenario: effect of age

With age the ability to understand speech in multitalker environments usually deteriorates. The central auditory system has to perceptually segregate and group the acoustic input into sequences of distinct auditory objects. The present study used electrophysiological measures to study effects of age on auditory stream segregation in a multitalker scenario. Younger and older adults were presented with streams of short speech stimuli. When a single target stream was presented, the occurrence of a rare (deviant) syllable among a frequent (standard) syllable elicited the mismatch negativity (MMN), an electrophysiological correlate of automatic deviance detection. The presence of a second, concurrent stream consisting of the deviant syllable of the target stream reduced the MMN amplitude, especially when located nearby the target stream. The decrease in MMN amplitude indicates that the rare syllable of the target stream was less perceived as deviant, suggesting reduced stream segregation with decreasing stream distance. Moreover, the presence of a concurrent stream increased the MMN peak latency of the older group but not that of the younger group. The results provide neurophysiological evidence for the effects of concurrent speech on auditory processing in older adults, suggesting that older adults need more time for stream segregation in the presence of concurrent speech.

Electrophysiological correlates of cocktail-party listening

Detecting, localizing, and selectively attending to a particular sound source of interest in complex auditory scenes composed of multiple competing sources is a remarkable capacity of the human auditory system. The neural basis of this so-called "cocktail-party effect" has remained largely unknown. Here, we studied the cortical network engaged in solving the "cocktail-party" problem, using event-related potentials (ERPs) in combination with two tasks demanding horizontal localization of a naturalistic target sound presented either in silence or in the presence of multiple competing sound sources. Presentation of multiple sound sources, as compared to single sources, induced an increased P1 amplitude, a reduction in N1, and a strong N2 component, resulting in a pronounced negativity in the ERP difference waveform (N2d) around 260 ms after stimulus onset. About 100 ms later, the anterior contralateral N2 subcomponent (N2ac) occurred in the multiple-sources condition, as computed from the amplitude difference for targets in the left minus right hemispaces. Cortical source analyses of the ERP modulation, resulting from the contrast of multiple vs. single sources, generally revealed an initial enhancement of electrical activity in right temporo-parietal areas, including auditory cortex, by multiple sources (at P1) that is followed by a reduction, with the primary sources shifting from right inferior parietal lobule (at N1) to left dorso-frontal cortex (at N2d). Thus, cocktail-party listening, as compared to single-source localization, appears to be based on a complex chronology of successive electrical activities within a specific cortical network involved in spatial hearing in complex situations.

Neural representation of three-dimensional acoustic space in the human temporal lobe

Sound localization is an important function of the human brain, but the underlying cortical mechanisms remain unclear. In this study, we recorded auditory stimuli in three-dimensional space and then replayed the stimuli through earphones during functional magnetic resonance imaging (fMRI). By employing a machine learning algorithm, we successfully decoded sound location from the blood oxygenation level-dependent signals in the temporal lobe. Analysis of the data revealed that different cortical patterns were evoked by sounds from different locations. Specifically, discrimination of sound location along the abscissa axis evoked robust responses in the left posterior superior temporal gyrus (STG) and right mid-STG, discrimination along the elevation (EL) axis evoked robust responses in the left posterior middle temporal lobe (MTL) and right STG, and discrimination along the ordinate axis evoked robust responses in the left mid-MTL and right mid-STG. These results support a distributed representation of acoustic space in human cortex.

Frequency-dependent auditory space representation in the human planum temporale

Functional magnetic resonance imaging (fMRI) findings suggest that a part of the planum temporale (PT) is involved in representing spatial properties of acoustic information. Here, we tested whether this representation of space is frequency-dependent or generalizes across spectral content, as required from high order sensory representations. Using sounds with two different spectral content and two spatial locations in individually tailored virtual acoustic environment, we compared three conditions in a sparse-fMRI experiment: Single Location, in which two sounds were both presented from one location; Fixed Mapping, in which there was one-to-one mapping between two sounds and two locations; and Mixed Mapping, in which the two sounds were equally likely to appear at either one of the two locations. We surmised that only neurons tuned to both location and frequency should be differentially adapted by the Mixed and Fixed mappings. Replicating our previous findings, we found adaptation to spatial location in the PT. Importantly, activation was higher for Mixed Mapping than for Fixed Mapping blocks, even though the two sounds and the two locations appeared equally in both conditions. These results show that spatially tuned neurons in the human PT are not invariant to the spectral content of sounds.

The effect of brain lesions on sound localization in complex acoustic environments

Localizing sound sources of interest in cluttered acoustic environments--as in the 'cocktail-party' situation--is one of the most demanding challenges to the human auditory system in everyday life. In this study, stroke patients' ability to localize acoustic targets in a single-source and in a multi-source setup in the free sound field were directly compared. Subsequent voxel-based lesion-behaviour mapping analyses were computed to uncover the brain areas associated with a deficit in localization in the presence of multiple distracter sound sources rather than localization of individually presented sound sources. Analyses revealed a fundamental role of the right planum temporale in this task. The results from the left hemisphere were less straightforward, but suggested an involvement of inferior frontal and pre- and postcentral areas. These areas appear to be particularly involved in the spectrotemporal analyses crucial for effective segregation of multiple sound streams from various locations, beyond the currently known network for localization of isolated sound sources in otherwise silent surroundings.

Interaction between dorsal and ventral processing streams: where, when and how?

The execution of complex visual, auditory, and linguistic behaviors requires a dynamic interplay between spatial ('where/how') and non-spatial ('what') information processed along the dorsal and ventral processing streams. However, while it is acknowledged that there must be some degree of interaction between the two processing networks, how they interact, both anatomically and functionally, is a question which remains little explored. The current review examines the anatomical, temporal, and behavioral evidence regarding three potential models of dual stream interaction: (1) computations along the two pathways proceed independently and in parallel, reintegrating within shared target brain regions; (2) processing along the separate pathways is modulated by the existence of recurrent feedback loops; and (3) information is transferred directly between the two pathways at multiple stages and locations along their trajectories.

Neural correlates of sound localization in complex acoustic environments

Listening to and understanding people in a “cocktail-party situation” is a remarkable feature of the human auditory system. Here we investigated the neural correlates of the ability to localize a particular sound among others in an acoustically cluttered environment with healthy subjects. In a sound localization task, five different natural sounds were presented from five virtual spatial locations during functional magnetic resonance imaging (fMRI). Activity related to auditory stream segregation was revealed in posterior superior temporal gyrus bilaterally, anterior insula, supplementary motor area, and frontoparietal network. Moreover, the results indicated critical roles of left planum temporale in extracting the sound of interest among acoustical distracters and the precuneus in orienting spatial attention to the target sound. We hypothesized that the left-sided lateralization of the planum temporale activation is related to the higher specialization of the left hemisphere for analysis of spectrotemporal sound features. Furthermore, the precuneus − a brain area known to be involved in the computation of spatial coordinates across diverse frames of reference for reaching to objects − seems to be also a crucial area for accurately determining locations of auditory targets in an acoustically complex scene of multiple sound sources. The precuneus thus may not only be involved in visuo-motor processes, but may also subserve related functions in the auditory modality.