Visual emotional context modulates brain potentials elicited by unattended tones

Lower Childhood Socioeconomic Status Is Associated with Greater Neural Responses to Ambient Auditory Changes in Adulthood

Humans' early life experience varies by socioeconomic status (SES), raising the question of how this difference is reflected in the adult brain. An important aspect of brain function is the ability to detect salient ambient changes while focusing on a task. Here, we ask whether subjective social status during childhood is reflected by the way young adults' brain detecting changes in irrelevant information. In two studies (total n = 58), we examine electrical brain responses in the frontocentral region to a series of auditory tones, consisting of standard stimuli (80%) and deviant stimuli (20%) interspersed randomly, while participants were engaged in various visual tasks. Both studies showed stronger automatic change detection indexed by MMN in lower SES individuals, regardless of the unattended sound's feature, attended emotional content, or study type. Moreover, we observed a larger MMN in lower-SES participants, although they did not show differences in brain and behavior responses to the attended task. Lower-SES people also did not involuntarily orient more attention to sound changes (i.e., deviant stimuli), as indexed by the P3a. The study indicates that individuals with lower subjective social status may have an increased ability to automatically detect changes in their environment, which may suggest their adaptation to their childhood environments.

Affective processing in dysphoria: Evidence from startle probe modulation of ERPs

The hypoactivation of the appetitive and defensive motivational systems in the brain is a feature of depression and might also represent a vulnerability factor for the disorder. A measure that can be employed to investigate both motivational systems is the electroencephalographic response to an acoustic startle probe during affective processing. Particularly, the amplitude of auditory event-related potentials (ERPs) components to the startle probe is smaller when the emotional context is more arousing. Neural responses to an unattended startle probe during an emotional passive viewing task of pleasant, neutral, and unpleasant pictures was employed to assess the activation of the approach and defensive motivational systems in a sample of individuals with (n = 24, 23 females) vs. without (n = 24, 23 females) dysphoria. The group without dysphoria showed a reduced startle-elicited N200 only in the context of pleasant relative to neutral pictures, indicating that the affective processing of the appetitive context might reduce the attentional resources needed to orient attention toward unattended non-salient stimuli. Conversely, the N200 amplitude was not attenuated for pleasant relative to neutral and unpleasant contexts in the group with dysphoria. Moreover, no within- or between-group differences emerged in the P300 amplitude. Taken together, the results of this study showed that depression vulnerability is characterized by reduced attention to pleasant contexts, suggesting a blunted affective processing of appetitive emotional stimuli.

Visually Perceived Negative Emotion Enhances Mismatch Negativity but Fails to Compensate for Age-Related Impairments

Objective: Automatic detection of auditory stimuli, represented by the mismatch negativity (MMN), facilitates rapid processing of salient stimuli in the environment. The amplitude of MMN declines with ageing. However, whether automatic detection of auditory stimuli is affected by visually perceived negative emotions with normal ageing remains unclear. We aimed to evaluate how fearful facial expressions affect the MMN amplitude under ageing.

Methods: We used a modified oddball paradigm to analyze the amplitude of N100 (N1) and MMN in 22 young adults and 21 middle-aged adults.

Results: We found that the amplitude of N1 elicited by standard tones was smaller under fearful facial expressions than neutral facial expressions and was more negative for young adults than middle-aged adults. The MMN amplitude under fearful facial expressions was greater than neutral facial expressions, but the amplitude in middle-aged adults was smaller than in young adults.

Conclusion: Visually perceived negative emotion promotes the extraction of auditory features. Additionally, it enhances the effect of auditory change detection in middle-aged adults but fails to compensate for this decline with normal ageing.

Significance: The study may help to understand how visually perceived emotion affects the early stage of auditory information processing from an event process perspective.

Sound gaps boost emotional audiovisual integration independent of attention: Evidence from an ERP study

The emotion discrimination paradigm was adopted to study the effect of interrupted sound on visual emotional processing under different attentional states. There were two experiments: Experiment 1: judging facial expressions (explicit task), Experiment 2: judging the position of a bar (implicit task). In Experiment 1, ERP results showed that there was a sound gap accelerating the effect of P1 present only under neutral faces. In Experiment 2, the accelerating effect (P1) existed regardless of the emotional condition. Combining two experiments, P1 findings suggest that sound gap enhances bottom-up attention. The N170 and late positive component (LPC) were found to be regulated by emotion face in both experiments, with fear over the neutral. Comparing the two experiments, the explicit task induced a larger LPC than the implicit task. Overall, sound gaps boosted the audiovisual integration by bottom-up attention in early integration, while cognitive expectations led to top-down attention in late stages.

Pupillary response reflects attentional modulation to sound after emotional arousal

There have been various studies on the effects of emotional visual processing on subsequent non-emotional auditory stimuli. A previous study with EEG has shown that responses to deviant sounds presented after presenting negative pictures collected more attentional resources than those for neutral pictures. To investigate such a compelling between emotional and cognitive processing, this study aimed to examined pupillary responses to an auditory stimulus after a positive, negative, or neutral emotional state was elicited by an emotional image. An emotional image was followed by a beep sound that was either repetitive or unexpected, and the pupillary dilation was measured. As a result, we found that the early component of the pupillary response to the beep sound was larger for negative and positive emotional states than the neutral emotional state, whereas the late component was larger for the positive emotional state than the negative and neutral emotional states. In addition, the peak latency of the pupillary response was earlier for negative than neutral or positive images. Further, to compensate for the disadvantage of low-temporal resolution of the pupillary data, the pupillary responses were deconvoluted and used in the analysis. The deconvolution analysis of pupillary responses confirmed that the responses to beep sound were more likely to be modulated by the emotional state rather than being influenced by the short presentation interval between the images and sounds. These findings suggested that pupil size index modulations in the compelling situation between emotional and cognitive processing.

Effects of one-pedal automobile operation on the driver's emotional state and cognitive workload

A one-pedal system for operating an electric vehicle allows drivers to flexibly accelerate and decelerate (and even stop) by using just an accelerator pedal. Based on previous findings, one-pedal operation is considered to have the potential to increase positive emotions and decrease cognitive workload. To test this possibility, the present study compared the emotional state and cognitive workload between one-pedal and conventional two-pedal operation. Participants drove a vehicle on public roads, and driving enjoyment (i.e., pleasure and immersion) and the cognitive workload (i.e., ease and effortlessness) were assessed by means of questionnaires. In addition, physiological variations associated with driving pleasure and difficulty were assessed by electroencephalography (EEG). Both the questionnaire and EEG results revealed an increase in driving enjoyment in one-pedal operation. On the other hand, only the EEG results suggested a decrease in the cognitive workload in one-pedal operation; the questionnaire results did not show a significant difference between the pedal conditions. These findings support the notion that one-pedal operation has a positive influence on the driver's mental state, though its influence on the cognitive workload will require further investigation. We discuss future directions toward a better understanding of the effects of one-pedal operation on the driver's mental state.

Losses tune differently than gains: how gains and losses shape attentional scope and influence goal pursuit

Research on the asymmetric effect of negative versus positive affective states (induced by gains or losses) on scope of attention, both at a perceptual and a conceptual level, is abundant. However, little is known about the moderating effect of anticipating gains or losses versus actually experiencing them and about any downstream consequences of these effects on goal-directed behaviour. In two studies, we show that gains versus losses induce qualitatively different processes. In Experiment 1, we demonstrate that the anticipation of monetary gains results in a narrowing of attentional scope, while experiencing gains broadens the scope of attention. We find the reverse pattern concerning monetary losses - while anticipation of monetary losses results in broadening of attentional scope, the actual experience of losses results in narrowing of attentional scope. Additionally, Experiment 2 replicates these findings and shows how differential attentional tuning as a function of the anticipation versus experience of gains versus losses modulates priming-induced goal-directed behaviour.

Does emotion change auditory prediction and deviance detection?

In the last decades, a growing number of studies provided compelling evidence supporting the interplay of cognitive and affective processes. However, it remains to be clarified whether and how an emotional context affects the prediction and detection of change in unattended sensory events. In an event-related potential (ERP) study, we probed the modulatory role of pleasant, unpleasant and neutral visual contexts on the brain response to automatic detection of change in spectral (intensity) vs. temporal (duration) sound features. Twenty participants performed a passive auditory oddball task. Additionally, we tested the relationship between ERPs and self-reported mood. Participants reported more negative mood after the negative block. The P2 amplitude elicited by standards was increased in a positive context. Mismatch Negativity (MMN) amplitude was decreased in the negative relative to the neutral and positive contexts, and was associated with self-reported mood. These findings suggest that the detection of regularities in the auditory stream was facilitated in a positive context, whereas a negative visual context interfered with prediction error elicitation, through associated mood changes. Both ERP and behavioral effects highlight the intricate links between emotion, perception and cognitive processes.

Fear Spreading Across Senses: Visual Emotional Events Alter Cortical Responses to Touch, Audition, and Vision

Attention and perception are potentiated for emotionally significant stimuli, promoting efficient reactivity and survival. But does such enhancement extend to stimuli simultaneously presented across different sensory modalities? We used functional magnetic resonance imaging in humans to examine the effects of visual emotional signals on concomitant sensory inputs in auditory, somatosensory, and visual modalities. First, we identified sensory areas responsive to task-irrelevant tones, touches, or flickers, presented bilaterally while participants attended to either a neutral or a fearful face. Then, we measured whether these responses were modulated by the emotional content of the face. Sensory responses in primary cortices were enhanced for auditory and tactile stimuli when these appeared with fearful faces, compared with neutral, but striate cortex responses to the visual stimuli were reduced in the left hemisphere, plausibly as a consequence of sensory competition. Finally, conjunction and functional connectivity analyses identified 2 distinct networks presumably responsible for these emotional modulatory processes, involving cingulate, insular, and orbitofrontal cortices for the increased sensory responses, and ventrolateral prefrontal cortex for the decreased sensory responses. These results suggest that emotion tunes the excitability of sensory systems across multiple modalities simultaneously, allowing the individual to adaptively process incoming inputs in a potentially threatening environment.

Happy heart, smiling eyes: A systematic review of positive mood effects on broadening of visuospatial attention

Positive mood contributes to mental and physical wellbeing. The broaden-and-build theory (Fredrickson, 2001) proposed that the beneficial effects of positive mood on life quality result from attentional broadening. In this article, we systematically review (following PRISMA guidelines; Moher et al., 2009), a host of studies investigating the nature and extent of attentional changes triggered by the experience of positive mood, with a focus on vision. While several studies reported a broadening of attention, others found that positive mood led to a more diffuse information processing style. Positive mood appears to lessen attention selectivity in a way that is context-specific and bound to limitations. We propose a new framework in which we postulate that positive mood impacts the balance between internally and externally directed attention, through modulations of cognitive control processes, instead of broadening attention per se. This novel model is able to accommodate discrepant findings, seeks to translate the phenomenon of the so-called broadening of attention with positive mood into functional terms, and provides plausible neurobiological mechanisms underlying this effect, suggesting a crucial role of the anterior and posterior cingulate cortex in this interaction.

Magnetoencephalography study of the effect of attention modulation on somatosensory processing in patients with major depressive disorder

AIMS

Although affective and/or attention modulation of somatosensory processing has been well studied, the biological bases of somatic symptoms in patients with major depressive disorder (MDD) have rarely been examined. To elucidate changes in somatosensory processing underlying somatic symptoms in patients with MDD, we conducted a magnetoencephalography study of patients with MDD and healthy controls.

METHODS

After median nerve stimulation, somatosensory evoked fields (SEF) were recorded in 10 patients with MDD and 10 sex-, age-, and height-matched healthy volunteers under somatosensory attending, visually attending, and non-attending conditions. The latencies and magnitudes of N20m and P60m SEF were examined.

RESULTS

In the MDD group, P60m latency was significantly prolonged, irrespective of attention modulation, whereas N20m latency and root mean squares N20m and P60m amplitudes remained unchanged. Prolonged P60m latency negatively correlated with the somatosensory threshold, which was relatively high in the MDD group. Prolonged P60m latency also negatively correlated with a state of anxiety during the examination, but not with depressive symptoms or psychotropic medication.

CONCLUSIONS

These results suggested that patients with MDD experience dysfunction in somatosensory information processing, approximately 60 ms after stimuli, irrespective of attentional conditions.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.

Emotional valence and spatial congruency differentially modulate crossmodal processing: an fMRI study

Salient exogenous stimuli modulate attentional processes and lead to attention shifts-even across modalities and at a pre-attentive level. Stimulus properties such as hemispheric laterality and emotional valence influence processing, but their specific interaction in audio-visual attention paradigms remains ambiguous. We conducted an fMRI experiment to investigate the interaction of supramodal spatial congruency, emotional salience, and stimulus presentation side on neural processes of attention modulation. Emotionally neutral auditory deviants were presented in a dichotic listening oddball design. Simultaneously, visual target stimuli (schematic faces) were presented, which displayed either a negative or a positive emotion. These targets were presented in the left or in the right visual field and were either spatially congruent (valid) or incongruent (invalid) with the concurrent deviant auditory stimuli. According to our expectation we observed that deviant stimuli serve as attention-directing cues for visual target stimuli. Region-of-interest (ROI) analyses suggested differential effects of stimulus valence and spatial presentation on the hemodynamic response in bilateral auditory cortices. These results underline the importance of valence and presentation side for attention guidance by deviant sound events and may hint at a hemispheric specialization for valence and attention processing.

Phasic boosting of auditory perception by visual emotion

Emotionally negative stimuli boost perceptual processes. There is little known, however, about the timing of this modulation. The present study aims at elucidating the phasic effects of, emotional processing on auditory processing within subsequent time-windows of visual emotional, processing in humans. We recorded the electroencephalogram (EEG) while participants responded to a, discrimination task of faces with neutral or fearful expressions. A brief complex tone, which subjects, were instructed to ignore, was displayed concomitantly, but with different asynchronies respective to, the image onset. Analyses of the N1 auditory event-related potential (ERP) revealed enhanced brain, responses in presence of fearful faces. Importantly, this effect occurred at picture-tone asynchronies of, 100 and 150ms, but not when these were displayed simultaneously, or at 50ms or 200ms asynchrony. These results confirm the existence of a fast-operating crossmodal effect of visual emotion on auditory, processing, suggesting a phasic variation according to the time-course of emotional processing.

Emotional processing modulates attentional capture of irrelevant sound input in adolescents

The main goal of this study was to investigate how emotional processing modulates the allocation of attention to irrelevant background sound events in adolescence. We examined the effect of viewing positively and negatively valenced video clips on components of event-related brain potentials (ERPs), while irrelevant sounds were presented to the ears. All sounds evoked the P1, N1, P2, and N2 components. The infrequent, randomly occurring novel environmental sounds evoked the P3a component in all trial types. The main finding was that the P3a component was larger in amplitude when evoked by salient, distracting background sound events when participants were watching negatively charged video clips, compared to when viewing of the positive or neutral video clips. The results suggest that the threshold for involuntary attention to the novel sounds was lowered during viewing of the negative movie contexts. This indicates a survival mechanism, which would be needed for more automatic processing of irrelevant sounds to monitor the unattended environment in situations perceived as more threatening.