Phasic boosting of auditory perception by visual emotion

The emotion of sound target modulates the auditory gaze cueing effect

The auditory gaze cueing effect (auditory-GCE) is a faster response to auditory targets at an eye-gaze cue location than at a non-cue location. Previous research has found that auditory-GCE can be influenced by the integration of both gaze direction and emotion conveyed through facial expressions. However, it is unclear whether the emotional information of auditory targets can be cross-modally integrated with gaze direction to affect auditory-GCE. Here, we set neutral faces with different gaze directions as cues and three emotional sounds (fearful, happy, and neutral) as targets to investigate how the emotion of sound target modulates the auditory-GCE. Moreover, we conducted a controlled experiment using arrow cues. The results show that the emotional content of sound targets influences the auditory-GCE but only for those induced by facial cues. Specifically, fearful sounds elicit a significantly larger auditory-GCE compared to happy and neutral sounds, indicating that the emotional content of auditory targets plays a modulating role in the auditory-GCE. Furthermore, this modulation appears to occur only at a higher level of social meaning, involving the integration of emotional information from a sound with social gaze direction, rather than at a lower level, which involves the integration of direction and auditory emotion.

Fear Spreading Across Senses: Visual Emotional Events Alter Cortical Responses to Touch, Audition, and Vision

Attention and perception are potentiated for emotionally significant stimuli, promoting efficient reactivity and survival. But does such enhancement extend to stimuli simultaneously presented across different sensory modalities? We used functional magnetic resonance imaging in humans to examine the effects of visual emotional signals on concomitant sensory inputs in auditory, somatosensory, and visual modalities. First, we identified sensory areas responsive to task-irrelevant tones, touches, or flickers, presented bilaterally while participants attended to either a neutral or a fearful face. Then, we measured whether these responses were modulated by the emotional content of the face. Sensory responses in primary cortices were enhanced for auditory and tactile stimuli when these appeared with fearful faces, compared with neutral, but striate cortex responses to the visual stimuli were reduced in the left hemisphere, plausibly as a consequence of sensory competition. Finally, conjunction and functional connectivity analyses identified 2 distinct networks presumably responsible for these emotional modulatory processes, involving cingulate, insular, and orbitofrontal cortices for the increased sensory responses, and ventrolateral prefrontal cortex for the decreased sensory responses. These results suggest that emotion tunes the excitability of sensory systems across multiple modalities simultaneously, allowing the individual to adaptively process incoming inputs in a potentially threatening environment.

Modulation of auditory spatial attention by visual emotional cues: differential effects of attentional engagement and disengagement for pleasant and unpleasant cues

Previous research has demonstrated that threatening, compared to neutral pictures, can bias attention towards non-emotional auditory targets. Here we investigated which subcomponents of attention contributed to the influence of emotional visual stimuli on auditory spatial attention. Participants indicated the location of an auditory target, after brief (250 ms) presentation of a spatially non-predictive peripheral visual cue. Responses to targets were faster at the location of the preceding visual cue, compared to at the opposite location (cue validity effect). The cue validity effect was larger for targets following pleasant and unpleasant cues compared to neutral cues, for right-sided targets. For unpleasant cues, the crossmodal cue validity effect was driven by delayed attentional disengagement, and for pleasant cues, it was driven by enhanced engagement. We conclude that both pleasant and unpleasant visual cues influence the distribution of attention across modalities and that the associated attentional mechanisms depend on the valence of the visual cue.