Attention relieves visual crowding: Dissociable effects of peripheral and central cues

Visual crowding can be reduced when attention is directed to the target by peripheral cues. However, it is unclear whether central cues relieve visual crowding to the same extent as peripheral cues. In this study, we combined the Posner cueing task and the crowding task to investigate the effect of exogenous and endogenous attention on crowding. In Experiment 1, five different stimulus-onset asychronies (SOAs) between the cue and the target and a predictive validity of 100% were adopted. Both attentional cues were shown to significantly reduce the effect of visual crowding, but the peripheral cue was more effective than the central cue. Furthermore, peripheral cues started to relieve visual crowding at the shortest SOA (100 ms), whereas central cues worked only at later SOAs (275 ms or above). When the predictive validity of the cue was decreased to 70% in Experiment 2, similar results to Experiment 1 were found, but the valid cue was less effective in reducing crowding than that in Experiment 1. In Experiment 3, when the predictive validity was decreased to 50%, a valid peripheral cue improved performance but a valid central cue did not, suggesting that endogenous attention but not exogenous attention can be voluntarily controlled when the cues are not predictive of the target's location. These findings collectively suggest that both peripheral and central cues can alleviate crowding, but they differ in terms of strength, time dynamics, and flexibility of voluntary control.

Spatial attention is not affected by alpha or beta transcranial alternating current stimulation: A registered report

Using Electroencephalography (EEG) an event-related change in alpha activity has been observed over primary sensory cortices during the allocation of spatial attention. This is most prominent during top-down, or endogenous, attention, and nearly absent in bottom-up, or exogenous orienting. These changes are highly lateralised, such that an increase in alpha power is seen ipsilateral to the attended region of space and a decrease is seen contralaterally. Whether these changes in alpha oscillatory activity are causally related to attentional resources, or to perceptual processes, or are simply epiphenomenal, is unknown. If alpha oscillations are indicative of a causal mechanism whereby attention is allocated to a region of space, it remains an open question as to whether this is driven by ipsilateral increases or contralateral decreases in alpha power. This preregistered report set out to test these questions. To do so, we used transcranial Alternating Current Stimulation (tACS) to modulate alpha activity in the somatosensory cortex whilst measuring performance on established tactile attention paradigms. All participants completed an endogenous and exogenous tactile attention task in three stimulation conditions; alpha, sham and beta. Sham and beta stimulation operated as controls so that any observed effects could be attributed to alpha stimulation specifically. We replicated previous behavioural findings in all stimulation conditions showing a facilitation of cued trials in the endogenous task, and inhibition of return in the exogenous task. However, these were not affected by stimulation manipulations. Using Bayes-factor analysis we show strong support for the null hypotheses - that the manipulation of Alpha by tACS does not cause changes in tactile spatial attention. This well-powered study, conducted over three separate days, is an important contribution to the current debate regarding the efficiency of brain stimulation.

Audiospatial evoked potentials for the assessment of spatial attention deficits in patients with severe cerebrovascular accidents

INTRODUCTION

Neuropsychological assessment of spatial orientation in post-acute patients with large brain lesions is often limited due to additional cognitive disorders like aphasia, apraxia, or reduced responsiveness.

METHODS

To cope with these limitations, we developed a paradigm using passive audiospatial event-related potentials (pAERPs): Participants were requested to merely listen over headphones to horizontally moving tones followed by a short tone ("target"), presented either on the side to which the cue moved or on the opposite side. Two runs of 120 trials were presented and we registered AERPs with two electrodes, mounted at C3 and C4. Nine sub-acute patients with large left hemisphere (LH) or right hemisphere (RH) lesions and nine controls participated.

RESULTS

Patients had no problems completing the assessment. RH patients showed a reduced N100 for left-sided targets in all conditions. LH patients showed a diminished N100 for invalid trials and contralesional targets.

CONCLUSION

Measuring AERPs for moving auditory cues and with two electrodes allows investigating spatial attentional deficits in patients with large RH and LH lesions, who are often unable to perform clinical tests. Our procedure can be implemented easily in an acute and rehabilitation setting and might enable investigating spatial attentional processes even in patients with minimal conscious awareness.

New insights on the ventral attention network: Active suppression and involuntary recruitment during a bimodal task

Detection of unexpected, yet relevant events is essential in daily life. fMRI studies have revealed the involvement of the ventral attention network (VAN), including the temporo-parietal junction (TPJ), in such process. In this MEG study with 34 participants (17 women), we used a bimodal (visual/auditory) attention task to determine the neuronal dynamics associated with suppression of the activity of the VAN during top-down attention and its recruitment when information from the unattended sensory modality is involuntarily integrated. We observed an anticipatory power increase of alpha/beta oscillations (12-20 Hz, previously associated with functional inhibition) in the VAN following a cue indicating the modality to attend. Stronger VAN power increases were associated with better task performance, suggesting that the VAN suppression prevents shifting attention to distractors. Moreover, the TPJ was synchronized with the frontal eye field in that frequency band, indicating that the dorsal attention network (DAN) might participate in such suppression. Furthermore, we found a 12-20 Hz power decrease and enhanced synchronization, in both the VAN and DAN, when information between sensory modalities was congruent, suggesting an involvement of these networks when attention is involuntarily enhanced due to multisensory integration. Our results show that effective multimodal attentional allocation includes the modulation of the VAN and DAN through upper-alpha/beta oscillations. Altogether these results indicate that the suppressing role of alpha/beta oscillations might operate beyond sensory regions.

Crossmodal spatial distraction across the lifespan

The ability to resist distracting stimuli whilst voluntarily focusing on a task is fundamental to our everyday cognitive functioning. Here, we investigated how this ability develops, and thereafter declines, across the lifespan using a single task/experiment. Young children (5-7 years), older children (10-11 years), young adults (20-27 years), and older adults (62-86 years) were presented with complex visual scenes. Endogenous (voluntary) attention was engaged by having the participants search for a visual target presented on either the left or right side of the display. The onset of the visual scenes was preceded - at stimulus onset asynchronies (SOAs) of 50, 200, or 500 ms - by a task-irrelevant sound (an exogenous crossmodal spatial distractor) delivered either on the same or opposite side as the visual target, or simultaneously on both sides (cued, uncued, or neutral trials, respectively). Age-related differences were revealed, especially in the extreme age-groups, which showed a greater impact of crossmodal spatial distractors. Young children were highly susceptible to exogenous spatial distraction at the shortest SOA (50 ms), whereas older adults were distracted at all SOAs, showing significant exogenous capture effects during the visual search task. By contrast, older children and young adults' search performance was not significantly affected by crossmodal spatial distraction. Overall, these findings present a detailed picture of the developmental trajectory of endogenous resistance to crossmodal spatial distraction from childhood to old age and demonstrate a different efficiency in coping with distraction across the four age-groups studied.

Modality-specific and multisensory mechanisms of spatial attention and expectation

In our natural environment, the brain needs to combine signals from multiple sensory modalities into a coherent percept. Whereas spatial attention guides perceptual decisions by prioritizing processing of signals that are task-relevant, spatial expectations encode the probability of signals over space. Previous studies have shown that behavioral effects of spatial attention generalize across sensory modalities. However, because they manipulated spatial attention as signal probability over space, these studies could not dissociate attention and expectation or assess their interaction. In two experiments, we orthogonally manipulated spatial attention (i.e., task-relevance) and expectation (i.e., signal probability) selectively in one sensory modality (i.e., primary modality) (experiment 1: audition, experiment 2: vision) and assessed their effects on primary and secondary sensory modalities in which attention and expectation were held constant. Our results show behavioral effects of spatial attention that are comparable for audition and vision as primary modalities; however, signal probabilities were learned more slowly in audition, so that spatial expectations were formed later in audition than vision. Critically, when these differences in learning between audition and vision were accounted for, both spatial attention and expectation affected responses more strongly in the primary modality in which they were manipulated and generalized to the secondary modality only in an attenuated fashion. Collectively, our results suggest that both spatial attention and expectation rely on modality-specific and multisensory mechanisms.

The Intricate Interplay of Spatial Attention and Expectation: a Multisensory Perspective

Attention (i.e., task relevance) and expectation (i.e., signal probability) are two critical top-down mechanisms guiding perceptual inference. Attention prioritizes processing of information that is relevant for observers’ current goals. Prior expectations encode the statistical structure of the environment. Research to date has mostly conflated spatial attention and expectation. Most notably, the Posner cueing paradigm manipulates spatial attention using probabilistic cues that indicate where the subsequent stimulus is likely to be presented. Only recently have studies attempted to dissociate the mechanisms of attention and expectation and characterized their interactive (i.e., synergistic) or additive influences on perception. In this review, we will first discuss methodological challenges that are involved in dissociating the mechanisms of attention and expectation. Second, we will review research that was designed to dissociate attention and expectation in the unisensory domain. Third, we will review the broad field of crossmodal endogenous and exogenous spatial attention that investigates the impact of attention across the senses. This raises the critical question of whether attention relies on amodal or modality-specific mechanisms. Fourth, we will discuss recent studies investigating the role of both spatial attention and expectation in multisensory perception, where the brain constructs a representation of the environment based on multiple sensory inputs. We conclude that spatial attention and expectation are closely intertwined in almost all circumstances of everyday life. Yet, despite their intimate relationship, attention and expectation rely on partly distinct neural mechanisms: while attentional resources are mainly shared across the senses, expectations can be formed in a modality-specific fashion.

Distinct Neural Mechanisms of Spatial Attention and Expectation Guide Perceptual Inference in a Multisensory World

Spatial attention (i.e., task-relevance) and expectation (i.e., signal probability) are two critical top-down mechanisms guiding perceptual inference. Spatial attention prioritizes processing of information at task-relevant locations. Spatial expectations encode the statistical structure of the environment. An unresolved question is how the brain allocates attention and forms expectations in a multisensory environment, where task-relevance and signal probability over space can differ across sensory modalities. We used functional magnetic resonance imaging in human participants (female and male) to investigate whether the brain encodes task-relevance and signal probability over space separately or interactively across sensory modalities. In a novel multisensory paradigm, we manipulated spatial attention and expectation selectively in audition and assessed their effects on behavioral and neural responses to auditory and visual stimuli. Our results show that both auditory and visual stimuli increased activations in a right-lateralized frontoparietal system, when they were presented at locations that were task-irrelevant in audition. Yet, only auditory stimuli increased activations in the medial prefrontal cortex when presented at expected locations and in audiovisual and frontoparietal cortices signaling a prediction error when presented at unexpected locations. This dissociation in multisensory generalization for attention and expectation effects shows that the brain controls attentional resources interactively across the senses but encodes the statistical structure of the environment as spatial expectations independently for each sensory system. Our results demonstrate that spatial attention and expectation engage partly overlapping neural systems via distinct mechanisms to guide perceptual inference in a multisensory world.SIGNIFICANCE STATEMENT In our natural environment the brain is exposed to a constant influx of signals through all our senses. How does the brain allocate attention and form spatial expectations in this multisensory environment? Because observers need to respond to stimuli regardless of their sensory modality, they may allocate attentional resources and encode the probability of events jointly across the senses. This psychophysics and neuroimaging study shows that the brain controls attentional resources interactively across the senses via a frontoparietal system but encodes the statistical structure of the environment independently for each sense in sensory and frontoparietal areas. Thus, spatial attention and expectation engage partly overlapping neural systems via distinct mechanisms to guide perceptual inference in a multisensory world.

Left hemispatial neglect and overt orienting in naturalistic conditions: Role of high-level and stimulus-driven signals

Deficits of visuospatial orienting in brain-damaged patients affected by hemispatial neglect have been extensively investigated. Nonetheless, spontaneous spatial orienting in naturalistic conditions is still poorly understood. Here, we investigated the role played by top-down and stimulus-driven signals in overt spatial orienting of neglect patients during free-viewing of short videos portraying everyday life situations. In Experiment 1, we assessed orienting when meaningful visual events competed on the left and right side of space, and tested whether sensory salience on the two sides biased orienting. In Experiment 2, we examined whether the spatial alignment of visual and auditory signals modulates orienting. The results of Experiment 1 showed that in neglect patients severe deficits in contralesional orienting were restricted to viewing conditions with bilateral visual events competing for attentional capture. In contrast, orienting towards the contralesional side was largely spared when the videos contained a single event on the left side. In neglect patients the processing of stimulus-driven salience was relatively spared and helped orienting towards the left side when multiple events were present. Experiment 2 showed that sounds spatially aligned with visual events on the left side improved orienting towards the otherwise neglected hemispace. Anatomical scans indicated that neglect patients suffered grey and white matter damages primarily in the ventral frontoparietal cortex. This suggests that the improvement of contralesional orienting associated with visual salience and audiovisual spatial alignment may be due to processing in the relatively intact dorsal frontoparietal areas. Our data show that in naturalistic environments, the presence of multiple meaningful events is a major determinant of spatial orienting deficits in neglect patients, whereas the salience of visual signals and the spatial alignment between auditory and visual signals can counteract spatial orienting deficits. These results open new perspectives to develop novel rehabilitation strategies based on the use of naturalistic stimuli.

Neuronal interactions in areas of spatial attention reflect avoidance of disgust, but orienting to danger

For survival, it is necessary to attend quickly towards dangerous objects, but to turn away from something that is disgusting. We tested whether fear and disgust sounds direct spatial attention differently. Using fMRI, a sound cue (disgust, fear or neutral) was presented to the left or right ear. The cue was followed by a visual target (a small arrow) which was located on the same (valid) or opposite (invalid) side as the cue. Participants were required to decide whether the arrow pointed up- or downwards while ignoring the sound cue. Behaviorally, responses were faster for invalid compared to valid targets when cued by disgust, whereas the opposite pattern was observed for targets after fearful and neutral sound cues. During target presentation, activity in the visual cortex and IPL increased for targets invalidly cued with disgust, but for targets validly cued with fear which indicated a general modulation of activation due to attention. For the TPJ, an interaction in the opposite direction was observed, consistent with its role in detecting targets at unattended positions and in relocating attention. As a whole our results indicate that a disgusting sound directs spatial attention away from its location, in contrast to fearful and neutral sounds.

The Curious Incident of Attention in Multisensory Integration: Bottom-up vs. Top-down

The role attention plays in our experience of a coherent, multisensory world is still controversial. On the one hand, a subset of inputs may be selected for detailed processing and multisensory integration in a top-down manner, i.e., guidance of multisensory integration by attention. On the other hand, stimuli may be integrated in a bottom-up fashion according to low-level properties such as spatial coincidence, thereby capturing attention. Moreover, attention itself is multifaceted and can be describedviaboth top-down and bottom-up mechanisms. Thus, the interaction between attention and multisensory integration is complex and situation-dependent. The authors of this opinion paper are researchers who have contributed to this discussion from behavioural, computational and neurophysiological perspectives. We posed a series of questions, the goal of which was to illustrate the interplay between bottom-up and top-down processes in various multisensory scenarios in order to clarify the standpoint taken by each author and with the hope of reaching a consensus. Although divergence of viewpoint emerges in the current responses, there is also considerable overlap: In general, it can be concluded that the amount of influence that attention exerts on MSI depends on the current task as well as prior knowledge and expectations of the observer. Moreover stimulus properties such as the reliability and salience also determine how open the processing is to influences of attention.

Parietal cortex integrates contextual and saliency signals during the encoding of natural scenes in working memory

The Brief presentation of a complex scene entails that only a few objects can be selected, processed indepth, and stored in memory. Both low-level sensory salience and high-level context-related factors (e.g., the conceptual match/mismatch between objects and scene context) contribute to this selection process, but how the interplay between these factors affects memory encoding is largely unexplored. Here, during fMRI we presented participants with pictures of everyday scenes. After a short retention interval, participants judged the position of a target object extracted from the initial scene. The target object could be either congruent or incongruent with the context of the scene, and could be located in a region of the image with maximal or minimal salience. Behaviourally, we found a reduced impact of saliency on visuospatial working memory performance when the target was out-of-context. Encoding-related fMRI results showed that context-congruent targets activated dorsoparietal regions, while context-incongruent targets de-activated the ventroparietal cortex. Saliency modulated activity both in dorsal and ventral regions, with larger context-related effects for salient targets. These findings demonstrate the joint contribution of knowledge-based and saliency-driven attention for memory encoding, highlighting a dissociation between dorsal and ventral parietal regions.

Supramodal executive control of attention

The human attentional system can be subdivided into three functional networks of alerting, orienting, and executive control. Although these networks have been extensively studied in the visuospatial modality, whether the same mechanisms are deployed across different sensory modalities remains unclear. In this study we used the attention network test for the visuospatial modality, in addition to two auditory variants with spatial and frequency manipulations to examine cross-modal correlations between network functions. Results showed that among the visual and auditory tasks, the effects of executive control, but not effects of alerting and orienting, were significantly correlated. These findings suggest that while alerting and orienting functions rely more upon modality-specific processes, the executive control of attention coordinates complex behavior via supramodal mechanisms.

Forced to remember: when memory is biased by salient information

The last decades have seen a rapid growing in the attempt to understand the key factors involved in the internal memory representation of the external world. Visual salience have been found to provide a major contribution in predicting the probability for an item/object embedded in a complex setting (i.e., a natural scene) to be encoded and then remembered later on. Here I review the existing literature highlighting the impact of perceptual- (based on low-level sensory features) and semantics-related salience (based on high-level knowledge) on short-term memory representation, along with the neural mechanisms underpinning the interplay between these factors. The available evidence reveal that both perceptual- and semantics-related factors affect attention selection mechanisms during the encoding of natural scenes. Biasing internal memory representation, both perceptual and semantics factors increase the probability to remember high- to the detriment of low-saliency items. The available evidence also highlight an interplay between these factors, with a reduced impact of perceptual-related salience in biasing memory representation as a function of the increasing availability of semantics-related salient information. The neural mechanisms underpinning this interplay involve the activation of different portions of the frontoparietal attention control network. Ventral regions support the assignment of selection/encoding priorities based on high-level semantics, while the involvement of dorsal regions reflects priorities assignment based on low-level sensory features.

Functional connectivity of dorsal and ventral frontoparietal seed regions during auditory orienting

Our ability to refocus auditory attention is vital for even the most routine day-to-day activities. Shifts in auditory attention can be initiated "voluntarily," or triggered "involuntarily" by unexpected novel sound events. Here we employed psychophysiological interaction (PPI) analyses of auditory functional MRI data, to compare functional connectivity patterns of distinct frontoparietal cortex regions during cued voluntary vs. novelty-driven involuntary auditory attention shifting. Overall, our frontoparietal seed regions exhibited significant PPI increases with auditory cortex (AC) areas during both cued and novelty-driven orienting. However, significant positive PPI patterns associated with voluntary auditory attention (cue>novel task regressor), but mostly absent in analyses emphasizing involuntary orienting (novel>cue task regressor), were observed with seeds within the frontal eye fields (FEF), superior parietal lobule (SPL), and right supramarginal gyri (SMG). In contrast, significant positive PPIs associated selectively with involuntary orienting were observed between ACs and seeds within the bilateral anterior interior frontal gyri (IFG), left posterior IFG, SMG, and posterior cingulate cortices (PCC). We also found indices of lateralization of different attention networks: PPI increases selective to voluntary attention occurred primarily within right-hemispheric regions, whereas those related to involuntary orienting were more frequent with left-hemisphere seeds. In conclusion, despite certain similarities in PPI patterns across conditions, the more dorsal aspects of right frontoparietal cortex demonstrated wider connectivity during cued/voluntary attention shifting, whereas certain left ventral frontoparietal seeds were more widely connected during novelty-triggered/involuntary orienting. Our findings provide partial support for distinct attention networks for voluntary and involuntary auditory attention.

The role of the right temporoparietal junction in attention and social interaction as revealed by ALE meta-analysis

The right temporoparietal junction (rTPJ) is frequently associated with different capacities that to shift attention to unexpected stimuli (reorienting of attention) and to understand others' (false) mental state [theory of mind (ToM), typically represented by false belief tasks]. Competing hypotheses either suggest the rTPJ representing a unitary region involved in separate cognitive functions or consisting of subregions subserving distinct processes. We conducted activation likelihood estimation (ALE) meta-analyses to test these hypotheses. A conjunction analysis across ALE meta-analyses delineating regions consistently recruited by reorienting of attention and false belief studies revealed the anterior rTPJ, suggesting an overarching role of this specific region. Moreover, the anatomical difference analysis unravelled the posterior rTPJ as higher converging in false belief compared with reorienting of attention tasks. This supports the concept of an exclusive role of the posterior rTPJ in the social domain. These results were complemented by meta-analytic connectivity mapping (MACM) and resting-state functional connectivity (RSFC) analysis to investigate whole-brain connectivity patterns in task-constrained and task-free brain states. This allowed for detailing the functional separation of the anterior and posterior rTPJ. The combination of MACM and RSFC mapping showed that the posterior rTPJ has connectivity patterns with typical ToM regions, whereas the anterior part of rTPJ co-activates with the attentional network. Taken together, our data suggest that rTPJ contains two functionally fractionated subregions: while posterior rTPJ seems exclusively involved in the social domain, anterior rTPJ is involved in both, attention and ToM, conceivably indicating an attentional shifting role of this region.

Cross-modal decoupling in temporal attention

Prior studies have repeatedly reported behavioural benefits to events occurring at attended, compared to unattended, points in time. It has been suggested that, as for spatial orienting, temporal orienting of attention spreads across sensory modalities in a synergistic fashion. However, the consequences of cross-modal temporal orienting of attention remain poorly understood. One challenge is that the passage of time leads to an increase in event predictability throughout a trial, thus making it difficult to interpret possible effects (or lack thereof). Here we used a design that avoids complete temporal predictability to investigate whether attending to a sensory modality (vision or touch) at a point in time confers beneficial access to events in the other, non-attended, sensory modality (touch or vision, respectively). In contrast to previous studies and to what happens with spatial attention, we found that events in one (unattended) modality do not automatically benefit from happening at the time point when another modality is expected. Instead, it seems that attention can be deployed in time with relative independence for different sensory modalities. Based on these findings, we argue that temporal orienting of attention can be cross-modally decoupled in order to flexibly react according to the environmental demands, and that the efficiency of this selective decoupling unfolds in time.

Medial parietal cortex activation related to attention control involving alcohol cues

Automatic attentional engagement toward and disengagement from alcohol cues play a role in alcohol use and dependence. In the current study, social drinkers performed a spatial cueing task designed to evoke conflict between such automatic processes and task instructions, a potentially important task feature from the perspective of recent dual-process models of addiction. Subjects received instructions either to direct their attention toward pictures of alcoholic beverages, and away from non-alcohol beverages; or to direct their attention toward pictures of non-alcoholic beverages, and away from alcohol beverages. Instructions were varied per block. Activation in medial parietal cortex was found during "approach alcohol" versus "avoid-alcohol" blocks. This region is associated with the, possibly automatic, shifting of attention between stimulus features. Subjects thus appeared to shift attention away from certain features of alcoholic cues when attention had to be directed toward their location. Further, activation in voxels located close to this region was negatively correlated with riskier drinking behavior. A tentative interpretation of the results is that risky drinking may be associated with a reduced automatic tendency to shift attention away from potentially distracting task-irrelevant alcohol cues. Future study is needed to test this interpretation, and to further determine the role of medial posterior regions in automatic alcohol-related attentional processes in general.

Lateralized parietotemporal oscillatory phase synchronization during auditory selective attention

Based on the infamous left-lateralized neglect syndrome, one might hypothesize that the dominating right parietal cortex has a bilateral representation of space, whereas the left parietal cortex represents only the contralateral right hemispace. Whether this principle applies to human auditory attention is not yet fully clear. Here, we explicitly tested the differences in cross-hemispheric functional coupling between the intraparietal sulcus (IPS) and auditory cortex (AC) using combined magnetoencephalography (MEG), EEG, and functional MRI (fMRI). Inter-regional pairwise phase consistency (PPC) was analyzed from data obtained during dichotic auditory selective attention task, where subjects were in 10-s trials cued to attend to sounds presented to one ear and to ignore sounds presented in the opposite ear. Using MEG/EEG/fMRI source modeling, parietotemporal PPC patterns were (a) mapped between all AC locations vs. IPS seeds and (b) analyzed between four anatomically defined AC regions-of-interest (ROI) vs. IPS seeds. Consistent with our hypothesis, stronger cross-hemispheric PPC was observed between the right IPS and left AC for attended right-ear sounds, as compared to PPC between the left IPS and right AC for attended left-ear sounds. In the mapping analyses, these differences emerged at 7-13Hz, i.e., at the theta to alpha frequency bands, and peaked in Heschl's gyrus and lateral posterior non-primary ACs. The ROI analysis revealed similarly lateralized differences also in the beta and lower theta bands. Taken together, our results support the view that the right parietal cortex dominates auditory spatial attention.

The Contributions of Sensory Dominance and Attentional Bias to Cross-modal Enhancement of Visual Cortex Excitability

Approaching or looming sounds (L-sounds) have been shown to selectively increase visual cortex excitability [Romei, V., Murray, M. M., Cappe, C., & Thut, G. Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Current Biology, 19, 1799–1805, 2009]. These cross-modal effects start at an early, preperceptual stage of sound processing and persist with increasing sound duration. Here, we identified individual factors contributing to cross-modal effects on visual cortex excitability and studied the persistence of effects after sound offset. To this end, we probed the impact of different L-sound velocities on phosphene perception postsound as a function of individual auditory versus visual preference/dominance using single-pulse TMS over the occipital pole. We found that the boosting of phosphene perception by L-sounds continued for several tens of milliseconds after the end of the L-sound and was temporally sensitive to different L-sound profiles (velocities). In addition, we found that this depended on an individual's preferred sensory modality (auditory vs. visual) as determined through a divided attention task (attentional preference), but not on their simple threshold detection level per sensory modality. Whereas individuals with “visual preference” showed enhanced phosphene perception irrespective of L-sound velocity, those with “auditory preference” showed differential peaks in phosphene perception whose delays after sound-offset followed the different L-sound velocity profiles. These novel findings suggest that looming signals modulate visual cortex excitability beyond sound duration possibly to support prompt identification and reaction to potentially dangerous approaching objects. The observed interindividual differences favor the idea that unlike early effects this late L-sound impact on visual cortex excitability is influenced by cross-modal attentional mechanisms rather than low-level sensory processes.

Spatial orienting in complex audiovisual environments

Previous studies on crossmodal spatial orienting typically used simple and stereotyped stimuli in the absence of any meaningful context. This study combined computational models, behavioural measures and functional magnetic resonance imaging to investigate audiovisual spatial interactions in naturalistic settings. We created short videos portraying everyday life situations that included a lateralised visual event and a co-occurring sound, either on the same or on the opposite side of space. Subjects viewed the videos with or without eye-movements allowed (overt or covert orienting). For each video, visual and auditory saliency maps were used to index the strength of stimulus-driven signals, and eye-movements were used as a measure of the efficacy of the audiovisual events for spatial orienting. Results showed that visual salience modulated activity in higher-order visual areas, whereas auditory salience modulated activity in the superior temporal cortex. Auditory salience modulated activity also in the posterior parietal cortex, but only when audiovisual stimuli occurred on the same side of space (multisensory spatial congruence). Orienting efficacy affected activity in the visual cortex, within the same regions modulated by visual salience. These patterns of activation were comparable in overt and covert orienting conditions. Our results demonstrate that, during viewing of complex multisensory stimuli, activity in sensory areas reflects both stimulus-driven signals and their efficacy for spatial orienting; and that the posterior parietal cortex combines spatial information about the visual and the auditory modality.

An event-related FMRI study of exogenous orienting across vision and audition

The orienting of attention to the spatial location of sensory stimuli in one modality based on sensory stimuli presented in another modality (i.e., cross-modal orienting) is a common mechanism for controlling attentional shifts. The neuronal mechanisms of top-down cross-modal orienting have been studied extensively. However, the neuronal substrates of bottom-up audio-visual cross-modal spatial orienting remain to be elucidated. Therefore, behavioral and event-related functional magnetic resonance imaging (FMRI) data were collected while healthy volunteers (N = 26) performed a spatial cross-modal localization task modeled after the Posner cuing paradigm. Behavioral results indicated that although both visual and auditory cues were effective in producing bottom-up shifts of cross-modal spatial attention, reorienting effects were greater for the visual cues condition. Statistically significant evidence of inhibition of return was not observed for either condition. Functional results also indicated that visual cues with auditory targets resulted in greater activation within ventral and dorsal frontoparietal attention networks, visual and auditory "where" streams, primary auditory cortex, and thalamus during reorienting across both short and long stimulus onset asynchronys. In contrast, no areas of unique activation were associated with reorienting following auditory cues with visual targets. In summary, current results question whether audio-visual cross-modal orienting is supramodal in nature, suggesting rather that the initial modality of cue presentation heavily influences both behavioral and functional results. In the context of localization tasks, reorienting effects accompanied by the activation of the frontoparietal reorienting network are more robust for visual cues with auditory targets than for auditory cues with visual targets.

Effects of parietal TMS on visual and auditory processing at the primary cortical level -- a concurrent TMS-fMRI study

Accumulating evidence suggests that multisensory interactions emerge already at the primary cortical level. Specifically, auditory inputs were shown to suppress activations in visual cortices when presented alone but amplify the blood oxygen level-dependent (BOLD) responses to concurrent visual inputs (and vice versa). This concurrent transcranial magnetic stimulation-functional magnetic resonance imaging (TMS-fMRI) study applied repetitive TMS trains at no, low, and high intensity over right intraparietal sulcus (IPS) and vertex to investigate top-down influences on visual and auditory cortices under 3 sensory contexts: visual, auditory, and no stimulation. IPS-TMS increased activations in auditory cortices irrespective of sensory context as a result of direct and nonspecific auditory TMS side effects. In contrast, IPS-TMS modulated activations in the visual cortex in a state-dependent fashion: it deactivated the visual cortex under no and auditory stimulation but amplified the BOLD response to visual stimulation. However, only the response amplification to visual stimulation was selective for IPS-TMS, while the deactivations observed for IPS- and Vertex-TMS resulted from crossmodal deactivations induced by auditory activity to TMS sounds. TMS to IPS may increase the responses in visual (or auditory) cortices to visual (or auditory) stimulation via a gain control mechanism or crossmodal interactions. Collectively, our results demonstrate that understanding TMS effects on (uni)sensory processing requires a multisensory perspective.

Evoked and intrinsic asymmetries during auditory attention: implications for the contralateral and neglect models of functioning

Unlike the visual system, a direct mapping of extrapersonal space does not exist within human auditory cortex (AC). Thus, models (contralateral bias vs. neglect) of how auditory spatial attention is allocated remain debated, as does the role of hemispheric asymmetries. To further examine these questions, 27 participants completed an exogenous auditory orienting task while undergoing functional magnetic resonance imaging. Resting-state data were also collected to characterize intrinsic activity within the AC. Current results provide the first evidence of hemispheric specialization in the "where" (right secondary AC) auditory processing stream during both evoked (orienting task) and intrinsic (resting-state data) activity, suggesting that spontaneous and evoked activity may be synchronized by similar cortical hierarchies. Strong evidence for a contralateral bias model was observed during rapid deployment stages (facilitation) of auditory attention in bilateral AC. However, contralateral bias increased for left and decreased for right AC (neglect model) after longer stimulus onset asynchronies (inhibition of return), suggesting a role for higher-order cortical structures in modulating AC functioning. Prime candidates for attentional modulation include the frontoparietal network, which demonstrated right hemisphere lateralization across multiple attentional states.

Enhancing multisensory spatial orienting by brain polarization of the parietal cortex

Transcranial direct current stimulation (tDCS) is a noninvasive brain stimulation technique that induces polarity-specific excitability changes in the human brain, therefore altering physiological, perceptual and higher-order cognitive processes. Here we investigated the possibility of enhancing attentional orienting within and across different sensory modalities, namely visual and auditory, by polarization of the posterior parietal cortex (PPC), given the putative involvement of this area in both unisensory and multisensory spatial processing. In different experiments, we applied anodal or sham tDCS to the right PPC and, for control, anodal stimulation of the right occipital cortex. Using a redundant signal effect (RSE) task, we found that anodal tDCS over the right PPC significantly speeded up responses to contralateral targets, regardless of the stimulus modality. Furthermore, the effect was dependant on the nature of the audiovisual enhancement, being stronger when subserved by a probabilistic mechanism induced by blue visual stimuli, which probably involves processing in the PPC. Hence, up-regulating the level of excitability in the PPC by tDCS appears a successful approach for enhancing spatial orienting to unisensory and crossmodal stimuli. Moreover, audiovisual interactions mostly occurring at a cortical level can be selectively enhanced by anodal PPC tDCS, whereas multisensory integration of stimuli, which is also largely mediated at a subcortical level, appears less susceptible to polarization of the cortex.

The multifaceted interplay between attention and multisensory integration

Multisensory integration has often been characterized as an automatic process. Recent findings indicate that multisensory integration can occur across various stages of stimulus processing that are linked to, and can be modulated by, attention. Stimulus-driven, bottom-up mechanisms induced by crossmodal interactions can automatically capture attention towards multisensory events, particularly when competition to focus elsewhere is relatively low. Conversely, top-down attention can facilitate the integration of multisensory inputs and lead to a spread of attention across sensory modalities. These findings point to a more intimate and multifaceted interplay between attention and multisensory integration than was previously thought. We review developments in the current understanding of the interactions between attention and multisensory processing, and propose a framework that unifies previous, apparently discordant, findings.

Exogenous and endogenous spatial attention effects on visuospatial working memory

In this study, we investigate how exogenous and endogenous orienting of spatial attention affect visuospatial working memory (VSWM). Specifically, we focused on two attentional effects and their consequences on storage in VSWM, when exogenous (Experiment 1) or endogenous (Experiment 2) orienting cues were used. The first effect, known as the meridian effect, is given by a decrement in behavioural performance when spatial cues and targets are presented in locations separated by vertical and/or horizontal meridians. The second effect, known as the distance effect, is given by a decrement in the orienting effects as a function of the spatial distance between cues and targets. Our results revealed a dissociation between exogenous and endogenous orienting mechanisms in terms of both meridian and distance effects. We found that meridian crossing affects performance only when endogenous cues were used. Specifically, VSWM performance with endogenous cueing depended more on the number of meridian crossings than on distance between cue and target. By contrast, a U-shaped distance dependency was observed using exogenous cues. Our findings therefore suggest that exogenous and endogenous orienting mechanisms lead to different forms of attentional bias for storage in VSWM.