Oscillatory signatures of crossmodal congruence effects: An EEG investigation employing a visuotactile pattern matching paradigm

Multi-timescale neural dynamics for multisensory integration

Carrying out any everyday task, be it driving in traffic, conversing with friends or playing basketball, requires rapid selection, integration and segregation of stimuli from different sensory modalities. At present, even the most advanced artificial intelligence-based systems are unable to replicate the multisensory processes that the human brain routinely performs, but how neural circuits in the brain carry out these processes is still not well understood. In this Perspective, we discuss recent findings that shed fresh light on the oscillatory neural mechanisms that mediate multisensory integration (MI), including power modulations, phase resetting, phase-amplitude coupling and dynamic functional connectivity. We then consider studies that also suggest multi-timescale dynamics in intrinsic ongoing neural activity and during stimulus-driven bottom-up and cognitive top-down neural network processing in the context of MI. We propose a new concept of MI that emphasizes the critical role of neural dynamics at multiple timescales within and across brain networks, enabling the simultaneous integration, segregation, hierarchical structuring and selection of information in different time windows. To highlight predictions from our multi-timescale concept of MI, real-world scenarios in which multi-timescale processes may coordinate MI in a flexible and adaptive manner are considered.

Unravelling the multisensory learning advantage: Different patterns of within and across frequency-specific interactions drive uni- and multisensory neuroplasticity

In the field of learning theory and practice, the superior efficacy of multisensory learning over uni-sensory is well-accepted. However, the underlying neural mechanisms at the macro-level of the human brain remain largely unexplored. This study addresses this gap by providing novel empirical evidence and a theoretical framework for understanding the superiority of multisensory learning. Through a cognitive, behavioral, and electroencephalographic assessment of carefully controlled uni-sensory and multisensory training interventions, our study uncovers a fundamental distinction in their neuroplastic patterns. A multilayered network analysis of pre- and post- training EEG data allowed us to model connectivity within and across different frequency bands at the cortical level. Pre-training EEG analysis unveils a complex network of distributed sources communicating through cross-frequency coupling, while comparison of pre- and post-training EEG data demonstrates significant differences in the reorganizational patterns of uni-sensory and multisensory learning. Uni-sensory training primarily modifies cross-frequency coupling between lower and higher frequencies, whereas multisensory training induces changes within the beta band in a more focused network, implying the development of a unified representation of audiovisual stimuli. In combination with behavioural and cognitive findings this suggests that, multisensory learning benefits from an automatic top-down transfer of training, while uni-sensory training relies mainly on limited bottom-up generalization. Our findings offer a compelling theoretical framework for understanding the advantage of multisensory learning.

Distinct mechanisms underlying cross-modal semantic conflict and response conflict processing

Interference from task-irrelevant stimuli can occur during the semantic and response processing stages. Previous studies have shown both common and distinct mechanisms underlying semantic conflict processing and response conflict processing in the visual domain. However, it remains unclear whether common and/or distinct mechanisms are involved in semantic conflict processing and response conflict processing in the cross-modal domain. Therefore, the present electroencephalography study adopted an audiovisual 2-1 mapping Stroop task to investigate whether common and/or distinct mechanisms underlie semantic conflict and response conflict. Behaviorally, significant cross-modal semantic conflict and significant cross-modal response conflict were observed. Electroencephalography results revealed that the frontal N2 amplitude and theta power increased only in the semantic conflict condition, while the parietal N450 amplitude increased only in the response conflict condition. These findings indicated that distinct neural mechanisms were involved in cross-modal semantic conflict and response conflict processing, supporting the domain-specific cognitive control mechanisms from a cross-modal multistage conflict processing perspective.

Temporal Electroencephalography Traits Dissociating Tactile Information and Cross-Modal Congruence Effects

To explore whether temporal electroencephalography (EEG) traits can dissociate the physical properties of touching objects and the congruence effects of cross-modal stimuli, we applied a machine learning approach to two major temporal domain EEG traits, event-related potential (ERP) and somatosensory evoked potential (SEP), for each anatomical brain region. During a task in which participants had to identify one of two material surfaces as a tactile stimulus, a photo image that matched ('congruent') or mismatched ('incongruent') the material they were touching was given as a visual stimulus. Electrical stimulation was applied to the median nerve of the right wrist to evoke SEP while the participants touched the material. The classification accuracies using ERP extracted in reference to the tactile/visual stimulus onsets were significantly higher than chance levels in several regions in both congruent and incongruent conditions, whereas SEP extracted in reference to the electrical stimulus onsets resulted in no significant classification accuracies. Further analysis based on current source signals estimated using EEG revealed brain regions showing significant accuracy across conditions, suggesting that tactile-based object recognition information is encoded in the temporal domain EEG trait and broader brain regions, including the premotor, parietal, and somatosensory areas.

Assessment of EEG-based functional connectivity in response to haptic delay

Haptic technologies enable users to physically interact with remote or virtual environments by applying force, vibration, or motion via haptic interfaces. However, the delivery of timely haptic feedback remains a challenge due to the stringent computation and communication requirements associated with haptic data transfer. Haptic delay disrupts the realism of the user experience and interferes with the quality of interaction. Research efforts have been devoted to studying the neural correlates of delayed sensory stimulation to better understand and thus mitigate the impact of delay. However, little is known about the functional neural networks that process haptic delay. This paper investigates the underlying neural networks associated with processing haptic delay in passive and active haptic interactions. Nineteen participants completed a visuo-haptic task using a computer screen and a haptic device while electroencephalography (EEG) data were being recorded. A combined approach based on phase locking value (PLV) functional connectivity and graph theory was used. To assay the effects of haptic delay on functional connectivity, we evaluate a global connectivity property through the small-worldness index and a local connectivity property through the nodal strength index. Results suggest that the brain exhibits significantly different network characteristics when a haptic delay is introduced. Haptic delay caused an increased manifestation of the small-worldness index in the delta and theta bands as well as an increased nodal strength index in the middle central region. Inter-regional connectivity analysis showed that the middle central region was significantly connected to the parietal and occipital regions as a result of haptic delay. These results are expected to indicate the detection of conflicting visuo-haptic information at the middle central region and their respective resolution and integration at the parietal and occipital regions.

Midfrontal theta power encodes the value of haptic delay

The use of haptic technologies in modern life scenarios is becoming the new normal particularly in rehabilitation, medical training, and entertainment applications. An evident challenge in haptic telepresence systems is the delay in haptic information. How humans perceive delayed visual and audio information has been extensively studied, however, the same for haptically delayed environments remains largely unknown. Here, we develop a visuo-haptic experimental setting that simulates pick and place task and involves continuous haptic feedback stimulation with four possible haptic delay levels. The setting is built using a haptic device and a computer screen. We use electroencephalography (EEG) to study the neural correlates that could be used to identify the amount of the experienced haptic delay. EEG data were collected from 34 participants. Results revealed that midfrontal theta oscillation plays a pivotal role in quantifying the amount of haptic delay while parietal alpha showed a significant modulation that encodes the presence of haptic delay. Based on the available literature, these results suggest that the amount of haptic delay is proportional to the neural activation that is associated with conflict detection and resolution as well as for multi-sensory divided attention.

Midfrontal theta oscillation encodes haptic delay

Haptic technologies aim to simulate tactile or kinesthetic interactions with a physical or virtual environment in order to enhance user experience and/or performance. However, due to stringent communication and computational needs, the user experience is influenced by delayed haptic feedback. While delayed feedback is well understood in the visual and auditory modalities, little research has systematically examined the neural correlates associated with delayed haptic feedback. In this paper, we used electroencephalography (EEG) to study sensory and cognitive neural correlates caused by haptic delay during passive and active tasks performed using a haptic device and a computer screen. Results revealed that theta power oscillation was significantly higher at the midfrontal cortex under the presence of haptic delay. Sensory correlates represented by beta rebound were found to be similar in the passive task and different in the active task under the delayed and synchronous conditions. Additionally, the event related potential (ERP) P200 component is modulated under the haptic delay condition during the passive task. The P200 amplitude significantly reduced in the last 20% of trials during the passive task and in the absence of haptic delay. Results suggest that haptic delay could be associated with increased cognitive control processes including multi-sensory divided attention followed by conflict detection and resolution with an earlier detection during the active task. Additionally, haptic delay tends to generate greater perceptual attention that does not significantly decay across trials during the passive task.

Reduced frontal white matter microstructure in healthy older adults with low tactile recognition performance

The aging of the nervous system is a heterogeneous process. It remains a significant challenge to identify relevant markers of pathological and healthy brain aging. A central aspect of aging are decreased sensory acuities, especially because they correlate with the decline in higher cognitive functioning. Sensory and higher cognitive processing relies on information flow between distant brain areas. Aging leads to disintegration of the underlying white matter tracts. While this disintegration is assumed to contribute to higher cognitive decline, data linking structural integrity and sensory function are sparse. The investigation of their interrelation may provide valuable insight into the mechanisms of brain aging. We used a combined behavioral and neuroimaging approach and investigated to what extent changes in microstructural white matter integrity reflect performance declines in tactile pattern recognition with aging. Poor performance in older participants was related to decreased integrity in the anterior corpus callosum. Probabilistic tractography showed that this structure is connected to the prefrontal cortices. Our data point to decreased integrity in the anterior corpus callosum as a marker for advanced brain aging. The correlation between impaired tactile recognition and disintegration in frontal brain networks could provide an explanation why the decrease of sensory function predicts cognitive decline.

New insights on the ventral attention network: Active suppression and involuntary recruitment during a bimodal task

Detection of unexpected, yet relevant events is essential in daily life. fMRI studies have revealed the involvement of the ventral attention network (VAN), including the temporo-parietal junction (TPJ), in such process. In this MEG study with 34 participants (17 women), we used a bimodal (visual/auditory) attention task to determine the neuronal dynamics associated with suppression of the activity of the VAN during top-down attention and its recruitment when information from the unattended sensory modality is involuntarily integrated. We observed an anticipatory power increase of alpha/beta oscillations (12-20 Hz, previously associated with functional inhibition) in the VAN following a cue indicating the modality to attend. Stronger VAN power increases were associated with better task performance, suggesting that the VAN suppression prevents shifting attention to distractors. Moreover, the TPJ was synchronized with the frontal eye field in that frequency band, indicating that the dorsal attention network (DAN) might participate in such suppression. Furthermore, we found a 12-20 Hz power decrease and enhanced synchronization, in both the VAN and DAN, when information between sensory modalities was congruent, suggesting an involvement of these networks when attention is involuntarily enhanced due to multisensory integration. Our results show that effective multimodal attentional allocation includes the modulation of the VAN and DAN through upper-alpha/beta oscillations. Altogether these results indicate that the suppressing role of alpha/beta oscillations might operate beyond sensory regions.

Sensory capability and information integration independently explain the cognitive status of healthy older adults

While there is evidence that sensory processing and multisensory integration change with age, links between these alterations and their relation to cognitive status remain unclear. In this study, we assessed sensory thresholds and performance of healthy younger and older adults in a visuotactile delayed match-to-sample task. Using Bayesian structural equation modelling (BSEM), we explored the factors explaining cognitive status in the group of older adults. Additionally, we applied transcranial alternating current stimulation (tACS) to a parieto-central network found to underlie visuotactile interactions and working memory matching in our previous work. Response times and signal detection measures indicated enhanced multisensory integration and enhanced benefit from successful working memory matching in older adults. Further, tACS caused a frequency-specific speeding (20 Hz) and delaying (70 Hz) of responses. Data exploration suggested distinct underlying factors for sensory acuity and sensitivity d’ on the one side, and multisensory and working memory enhancement on the other side. Finally, BSEM showed that these two factors labelled ‘sensory capability’ and ‘information integration’ independently explained cognitive status. We conclude that sensory decline and enhanced information integration might relate to distinct processes of ageing and discuss a potential role of the parietal cortex in mediating augmented integration in older adults.

Crossmodal Pattern Discrimination in Humans and Robots: A Visuo-Tactile Case Study

The quality of crossmodal perception hinges on two factors: The accuracy of the independent unimodal perception and the ability to integrate information from different sensory systems. In humans, the ability for cognitively demanding crossmodal perception diminishes from young to old age. Here, we propose a new approach to research to which degree the different factors contribute to crossmodal processing and the age-related decline by replicating a medical study on visuo-tactile crossmodal pattern discrimination utilizing state-of-the-art tactile sensing technology and artificial neural networks (ANN). We implemented two ANN models to specifically focus on the relevance of early integration of sensory information during the crossmodal processing stream as a mechanism proposed for efficient processing in the human brain. Applying an adaptive staircase procedure, we approached comparable unimodal classification performance for both modalities in the human participants as well as the ANN. This allowed us to compare crossmodal performance between and within the systems, independent of the underlying unimodal processes. Our data show that unimodal classification accuracies of the tactile sensing technology are comparable to humans. For crossmodal discrimination of the ANN the integration of high-level unimodal features on earlier stages of the crossmodal processing stream shows higher accuracies compared to the late integration of independent unimodal classifications. In comparison to humans, the ANN show higher accuracies than older participants in the unimodal as well as the crossmodal condition, but lower accuracies than younger participants in the crossmodal task. Taken together, we can show that state-of-the-art tactile sensing technology is able to perform a complex tactile recognition task at levels comparable to humans. For crossmodal processing, human inspired early sensory integration seems to improve the performance of artificial neural networks. Still, younger participants seem to employ more efficient crossmodal integration mechanisms than modeled in the proposed ANN. Our work demonstrates how collaborative research in neuroscience and embodied artificial neurocognitive models can help to derive models to inform the design of future neurocomputational architectures.

Multi-spectral oscillatory dynamics serving directed and divided attention

Attention-related amplification of neural representations of external stimuli has been well documented in the visual domain, however, research concerning the oscillatory dynamics of such directed attention is relatively sparse in humans. Specifically, it is unknown which spectrally-specific neural responses are mainly impacted by the direction and division of attention, as well as whether the effects of attention on these oscillations are spatially disparate. In this study, we use magnetoencephalography and a visual-somatosensory oddball task to investigate the whole-brain oscillatory dynamics of directed (Experiment 1; N = 26) and divided (Experiment 2; N = 34) visual attention. Sensor-level data were transformed into the time-frequency domain and significant responses from baseline were imaged using a frequency-resolved beamformer. We found that multi-spectral cortical oscillations were stronger when attention was sustained in the visual space and that these effects exhibited informative spatial distributions that differed by frequency. More specifically, we found stronger frontal theta (4–8 Hz), frontal and occipital alpha (8–14 Hz), occipital beta (16–22 Hz), and frontal gamma (74–84 Hz) responses when visual attention was sustained than when it was directed away from the visual domain. Similarly, in the divided attention condition, we observed stronger fronto-parietal theta activity and temporo-parietal alpha and beta oscillations when visual attention was sustained toward the visual stimuli than divided between the visual and somatosensory domains. Investigating how attentional gain is implemented in the human brain is essential for better understanding how this process is degraded in disease, and may provide useful targets for future therapies.

Crossmodal Congruency Enhances Performance of Healthy Older Adults in Visual-Tactile Pattern Matching

One of the pivotal challenges of aging is to maintain independence in the activities of daily life. In order to adapt to changes in the environment, it is crucial to continuously process and accurately combine simultaneous input from different sensory systems, i.e., crossmodal or multisensory integration. With aging, performance decreases in multiple domains, affecting bottom-up sensory processing as well as top-down control. However, whether this decline leads to impairments in crossmodal interactions remains an unresolved question. While some researchers propose that crossmodal interactions degrade with age, others suggest that they are conserved or even gain compensatory importance. To address this question, we compared the behavioral performance of older and young participants in a well-established crossmodal matching task, requiring the evaluation of congruency in simultaneously presented visual and tactile patterns. Older participants performed significantly worse than young controls in the crossmodal task when being stimulated at their individual unimodal visual and tactile perception thresholds. Performance increased with adjustment of stimulus intensities. This improvement was driven by better detection of congruent stimulus pairs, while the detection of incongruent pairs was not significantly enhanced. These results indicate that age-related impairments lead to poor performance in complex crossmodal scenarios and demanding cognitive tasks. Crossmodal congruency effects attenuate the difficulties of older adults in visuotactile pattern matching and might be an important factor to drive the benefits of older adults demonstrated in various crossmodal integration scenarios. Congruency effects might, therefore, be used to develop strategies for cognitive training and neurological rehabilitation.

The Evaluation of Creative Ideas in Older and Younger Adults

Abstract. The ability to solve problems of divergent type is one of the most intact functions in successful aging. However, neurophysiologic mechanisms that support the efficiency of creative thinking remain largely unknown. This study was aimed to investigate age-related difference in localized induced electroencephalogram (EEG) changes during creative idea evaluation stage of divergent problem-solving (Alternate Uses Task), using standardized low-resolution brain electromagnetic tomography. Younger (45 women, 44 men, Mage = 22.1 years, age range: 18–30 years) and older adults (46 women, 43 men, Mage = 64.9 years, age range: 55–75 years) participated in the study. Higher synchronization in individually adjusted theta frequency band [from (individual alpha peak frequency −6 Hz) to (individual alpha peak frequency −4 Hz)] in anterior areas with the maximum values in anterior cingulate gyrus was revealed in older as compared with younger participants by group contrast. Higher desynchronization in wide beta range [from (individual alpha peak frequency +2 Hz) to 30 Hz] was localized in posterior brain regions with the highest values in posterior cingulate gyrus, precuneus, and parietal lobule in older adults. Induced beta 2 synchronization was positively correlated with originality (as measured by the mean frequency of ideas) in younger and years of education in older subjects. Based on the data, it was supposed that controlling the decision-making processes is more important for older adults while maintenance of the internal image of elements’ recombination may play essential role for younger subjects.

Probing the "Default Network Interference Hypothesis" With EEG: An RDoC Approach Focused on Attention

Studies have shown that specific networks (default mode network [DMN] and task positive network [TPN]) activate in an anticorrelated manner when sustaining attention. Related EEG studies are scarce and often lack behavioral validation. We performed independent component analysis (ICA) across different frequencies (source-level), using eLORETA-ICA, to extract brain-network activity during resting-state and sustained attention. We applied ICA to the voxel domain, similar to functional magnetic resonance imaging methods of analyses. The obtained components were contrasted and correlated to attentional performance (omission errors) in a large sample of healthy subjects (N = 1397). We identified one component that robustly correlated with inattention and reflected an anticorrelation of delta activity in the anterior cingulate and precuneus, and delta and theta activity in the medial prefrontal cortex and with alpha and gamma activity in medial frontal regions. We then compared this component between optimal and suboptimal attentional performers. For the latter group, we observed a greater change in component loading between resting-state and sustained attention than for the optimal performers. Following the National Institute of Mental Health Research Domain Criteria (RDoC) approach, we prospectively replicated and validated these findings in subjects with attention deficit/hyperactivity disorder. Our results provide further support for the "default mode interference hypothesis."

Long-range functional coupling predicts performance: Oscillatory EEG networks in multisensory processing

The integration of sensory signals from different modalities requires flexible interaction of remote brain areas. One candidate mechanism to establish communication in the brain is transient synchronization of oscillatory neural signals. Although there is abundant evidence for the involvement of cortical oscillations in brain functions based on the analysis of local power, assessment of the phase dynamics among spatially distributed neuronal populations and their relevance for behavior is still sparse. In the present study, we investigated the interaction between remote brain areas by analyzing high-density electroencephalogram (EEG) data obtained from human participants engaged in a visuotactile pattern matching task. We deployed an approach for purely data-driven clustering of neuronal phase coupling in source space, which allowed imaging of large-scale functional networks in space, time and frequency without defining a priori constraints. Based on the phase coupling results, we further explored how brain areas interacted across frequencies by computing phase-amplitude coupling. Several networks of interacting sources were identified with our approach, synchronizing their activity within and across the theta (∼5 Hz), alpha (∼10 Hz), and beta (∼20 Hz) frequency bands and involving multiple brain areas that have previously been associated with attention and motor control. We demonstrate the functional relevance of these networks by showing that phase delays - in contrast to spectral power - were predictive of task performance. The data-driven analysis approach employed in the current study allowed an unbiased examination of functional brain networks based on EEG source level connectivity data. Showcased for multisensory processing, our results provide evidence that large-scale neuronal coupling is vital to long-range communication in the human brain and relevant for the behavioral outcome in a cognitive task.

Frontal and parietal alpha oscillations reflect attentional modulation of cross-modal matching

Multisensory perception is shaped by both attentional selection of relevant sensory inputs and exploitation of stimulus-driven factors that promote cross-modal binding. Underlying mechanisms of both top-down and bottom-up modulations have been linked to changes in alpha/gamma dynamics in primary sensory cortices and temporoparietal cortex. Accordingly, it has been proposed that alpha oscillations provide pulsed inhibition for gamma activity and thereby dynamically route cortical information flow. In this study, we employed a recently introduced multisensory paradigm incorporating both bottom-up and top-down aspects of cross-modal attention in an EEG study. The same trimodal stimuli were presented in two distinct attentional conditions, focused on visual-tactile or audio-visual components, for which cross-modal congruence of amplitude changes had to be evaluated. Neither top-down nor bottom-up cross-modal attention modulated alpha or gamma power in primary sensory cortices. Instead, we found alpha band effects in bilateral frontal and right parietal cortex. We propose that frontal alpha oscillations reflect the origin of top-down control regulating perceptual gains and that modulations of parietal alpha oscillations relates to intersensory re-orienting. Taken together, we suggest that the idea of selective cortical routing via alpha oscillations can be extended from sensory cortices to the frontoparietal attention network.

Neural Oscillations Orchestrate Multisensory Processing

At any given moment, we receive input through our different sensory systems, and this information needs to be processed and integrated. Multisensory processing requires the coordinated activity of distinct cortical areas. Key mechanisms implicated in these processes include local neural oscillations and functional connectivity between distant cortical areas. Evidence is now emerging that neural oscillations in distinct frequency bands reflect different mechanisms of multisensory processing. Moreover, studies suggest that aberrant neural oscillations contribute to multisensory processing deficits in clinical populations, such as schizophrenia. In this article, we review recent literature on the neural mechanisms underlying multisensory processing, focusing on neural oscillations. We derive a framework that summarizes findings on (1) stimulus-driven multisensory processing, (2) the influence of top-down information on multisensory processing, and (3) the role of predictions for the formation of multisensory perception. We propose that different frequency band oscillations subserve complementary mechanisms of multisensory processing. These processes can act in parallel and are essential for multisensory processing.

Ripe for solution: Delayed development of multisensory processing in autism and its remediation

Difficulty integrating inputs from different sensory sources is commonly reported in individuals with Autism Spectrum Disorder (ASD). Accumulating evidence consistently points to altered patterns of behavioral reactions and neural activity when individuals with ASD observe or act upon information arriving through multiple sensory systems. For example, impairments in the integration of seen and heard speech appear to be particularly acute, with obvious implications for interpersonal communication. Here, we explore the literature on multisensory processing in autism with a focus on developmental trajectories. While much remains to be understood, some consistent observations emerge. Broadly, sensory integration deficits are found in children with an ASD whereas these appear to be much ameliorated, or even fully recovered, in older teenagers and adults on the spectrum. This protracted delay in the development of multisensory processing raises the possibility of applying early intervention strategies focused on multisensory integration, to accelerate resolution of these functions. We also consider how dysfunctional cross-sensory oscillatory neural communication may be one key pathway to impaired multisensory processing in ASD.

Oscillatory activity in auditory cortex reflects the perceptual level of audio-tactile integration

Cross-modal interactions between sensory channels have been shown to depend on both the spatial disparity and the perceptual similarity between the presented stimuli. Here we investigate the behavioral and neural integration of auditory and tactile stimulus pairs at different levels of spatial disparity. Additionally, we modulated the amplitudes of both stimuli in either a coherent or non-coherent manner. We found that both auditory and tactile localization performance was biased towards the stimulus in the respective other modality. This bias linearly increases with stimulus disparity and is more pronounced for coherently modulated stimulus pairs. Analyses of electroencephalographic (EEG) activity at temporal–cortical sources revealed enhanced event-related potentials (ERPs) as well as decreased alpha and beta power during bimodal as compared to unimodal stimulation. However, while the observed ERP differences are similar for all stimulus combinations, the extent of oscillatory desynchronization varies with stimulus disparity. Moreover, when both stimuli were subjectively perceived as originating from the same direction, the reduction in alpha and beta power was significantly stronger. These observations suggest that in the EEG the level of perceptual integration is mainly reflected by changes in ongoing oscillatory activity.

Oscillatory brain activity during multisensory attention reflects activation, disinhibition, and cognitive control

In this study, we used a novel multisensory attention paradigm to investigate attention-modulated cortical oscillations over a wide range of frequencies using magnetencephalography in healthy human participants. By employing a task that required the evaluation of the congruence of audio-visual stimuli, we promoted the formation of widespread cortical networks including early sensory cortices as well as regions associated with cognitive control. We found that attention led to increased high-frequency gamma-band activity and decreased lower frequency theta-, alpha-, and beta-band activity in early sensory cortex areas. Moreover, alpha-band coherence decreased in visual cortex. Frontal cortex was found to exert attentional control through increased low-frequency phase synchronisation. Crossmodal congruence modulated beta-band coherence in mid-cingulate and superior temporal cortex. Together, these results offer an integrative view on the concurrence of oscillations at different frequencies during multisensory attention.

Attentional Resource Allocation in Visuotactile Processing Depends on the Task, But Optimal Visuotactile Integration Does Not Depend on Attentional Resources

Humans constantly process and integrate sensory input from multiple sensory modalities. However, the amount of input that can be processed is constrained by limited attentional resources. A matter of ongoing debate is whether attentional resources are shared across sensory modalities, and whether multisensory integration is dependent on attentional resources. Previous research suggested that the distribution of attentional resources across sensory modalities depends on the the type of tasks. Here, we tested a novel task combination in a dual task paradigm: Participants performed a self-terminated visual search task and a localization task in either separate sensory modalities (i.e., haptics and vision) or both within the visual modality. Tasks considerably interfered. However, participants performed the visual search task faster when the localization task was performed in the tactile modality in comparison to performing both tasks within the visual modality. This finding indicates that tasks performed in separate sensory modalities rely in part on distinct attentional resources. Nevertheless, participants integrated visuotactile information optimally in the localization task even when attentional resources were diverted to the visual search task. Overall, our findings suggest that visual search and tactile localization partly rely on distinct attentional resources, and that optimal visuotactile integration is not dependent on attentional resources.

Audition and vision share spatial attentional resources, yet attentional load does not disrupt audiovisual integration

Humans continuously receive and integrate information from several sensory modalities. However, attentional resources limit the amount of information that can be processed. It is not yet clear how attentional resources and multisensory processing are interrelated. Specifically, the following questions arise: (1) Are there distinct spatial attentional resources for each sensory modality? and (2) Does attentional load affect multisensory integration? We investigated these questions using a dual task paradigm: participants performed two spatial tasks (a multiple object tracking task and a localization task), either separately (single task condition) or simultaneously (dual task condition). In the multiple object tracking task, participants visually tracked a small subset of several randomly moving objects. In the localization task, participants received either visual, auditory, or redundant visual and auditory location cues. In the dual task condition, we found a substantial decrease in participants' performance relative to the results of the single task condition. Importantly, participants performed equally well in the dual task condition regardless of the location cues' modality. This result suggests that having spatial information coming from different modalities does not facilitate performance, thereby indicating shared spatial attentional resources for the auditory and visual modality. Furthermore, we found that participants integrated redundant multisensory information similarly even when they experienced additional attentional load in the dual task condition. Overall, findings suggest that (1) visual and auditory spatial attentional resources are shared and that (2) audiovisual integration of spatial information occurs in an pre-attentive processing stage.

A matter of attention: Crossmodal congruence enhances and impairs performance in a novel trimodal matching paradigm

A novel crossmodal matching paradigm including vision, audition, and somatosensation was developed in order to investigate the interaction between attention and crossmodal congruence in multisensory integration. To that end, all three modalities were stimulated concurrently while a bimodal focus was defined blockwise. Congruence between stimulus intensity changes in the attended modalities had to be evaluated. We found that crossmodal congruence improved performance if both, the attended modalities and the task-irrelevant distractor were congruent. If the attended modalities were incongruent, the distractor impaired performance due to its congruence relation to one of the attended modalities. Between attentional conditions, magnitudes of crossmodal enhancement or impairment differed. Largest crossmodal effects were seen in visual-tactile matching, intermediate effects for audio-visual and smallest effects for audio-tactile matching. We conclude that differences in crossmodal matching likely reflect characteristics of multisensory neural network architecture. We discuss our results with respect to the timing of perceptual processing and state hypotheses for future physiological studies. Finally, etiological questions are addressed.