Emotionally negative pictures increase attention to a subsequent auditory stimulus

Visually Perceived Negative Emotion Enhances Mismatch Negativity but Fails to Compensate for Age-Related Impairments

Objective: Automatic detection of auditory stimuli, represented by the mismatch negativity (MMN), facilitates rapid processing of salient stimuli in the environment. The amplitude of MMN declines with ageing. However, whether automatic detection of auditory stimuli is affected by visually perceived negative emotions with normal ageing remains unclear. We aimed to evaluate how fearful facial expressions affect the MMN amplitude under ageing.

Methods: We used a modified oddball paradigm to analyze the amplitude of N100 (N1) and MMN in 22 young adults and 21 middle-aged adults.

Results: We found that the amplitude of N1 elicited by standard tones was smaller under fearful facial expressions than neutral facial expressions and was more negative for young adults than middle-aged adults. The MMN amplitude under fearful facial expressions was greater than neutral facial expressions, but the amplitude in middle-aged adults was smaller than in young adults.

Conclusion: Visually perceived negative emotion promotes the extraction of auditory features. Additionally, it enhances the effect of auditory change detection in middle-aged adults but fails to compensate for this decline with normal ageing.

Significance: The study may help to understand how visually perceived emotion affects the early stage of auditory information processing from an event process perspective.

Representing the Good and Bad: fMRI signatures during the encoding of multisensory positive, negative, and neutral events

Few studies have examined how multisensory emotional experiences are processed and encoded into memory. Here, we aimed to determine whether, at encoding, activity within functionally-defined visual- and auditory-processing brain regions discriminated the emotional category (i.e., positive, negative, or neutral) of the multisensory (audio-visual) events. Participants incidentally encoded positive, negative, and neutral multisensory stimuli during event-related functional magnetic resonance imaging (fMRI). Following a 3-h post-encoding delay, their memory for studied stimuli was tested, allowing us to identify emotion-category-specific subsequent-memory effects focusing on medial temporal lobe regions (i.e., amygdala, hippocampus) and visual- and auditory-processing regions. We used a combination of univariate and multivoxel pattern fMRI analyses (MVPA) to examine emotion-category-specificity in mean activity levels and neural patterning, respectively. Univariate analyses revealed many more visual regions that showed negative-category-specificity relative to positive-category-specificity, and auditory regions only showed negative-category-specificity. These results suggest that negative emotion is more closely tied to information contained within sensory regions, a conclusion that was supported by the MVPA analyses. Functional connectivity analyses further revealed that the visual amplification of category-selective processing is driven, in part, by mean signal from the amygdala. Interestingly, while stronger representations in visuo-auditory regions were related to subsequent-memory for neutral multisensory stimuli, they were related to subsequent-forgetting of positive and negative stimuli. Neural patterning in the hippocampus and amygdala were related to memory for negative multisensory stimuli. These results provide new evidence that negative emotional stimuli are processed with increased engagement of visuosensory regions, but that this sensory engagement-that generalizes across the entire emotion category-is not the type of sensory encoding that is most beneficial for later retrieval.

Sound gaps boost emotional audiovisual integration independent of attention: Evidence from an ERP study

The emotion discrimination paradigm was adopted to study the effect of interrupted sound on visual emotional processing under different attentional states. There were two experiments: Experiment 1: judging facial expressions (explicit task), Experiment 2: judging the position of a bar (implicit task). In Experiment 1, ERP results showed that there was a sound gap accelerating the effect of P1 present only under neutral faces. In Experiment 2, the accelerating effect (P1) existed regardless of the emotional condition. Combining two experiments, P1 findings suggest that sound gap enhances bottom-up attention. The N170 and late positive component (LPC) were found to be regulated by emotion face in both experiments, with fear over the neutral. Comparing the two experiments, the explicit task induced a larger LPC than the implicit task. Overall, sound gaps boosted the audiovisual integration by bottom-up attention in early integration, while cognitive expectations led to top-down attention in late stages.

Pupillary response reflects attentional modulation to sound after emotional arousal

There have been various studies on the effects of emotional visual processing on subsequent non-emotional auditory stimuli. A previous study with EEG has shown that responses to deviant sounds presented after presenting negative pictures collected more attentional resources than those for neutral pictures. To investigate such a compelling between emotional and cognitive processing, this study aimed to examined pupillary responses to an auditory stimulus after a positive, negative, or neutral emotional state was elicited by an emotional image. An emotional image was followed by a beep sound that was either repetitive or unexpected, and the pupillary dilation was measured. As a result, we found that the early component of the pupillary response to the beep sound was larger for negative and positive emotional states than the neutral emotional state, whereas the late component was larger for the positive emotional state than the negative and neutral emotional states. In addition, the peak latency of the pupillary response was earlier for negative than neutral or positive images. Further, to compensate for the disadvantage of low-temporal resolution of the pupillary data, the pupillary responses were deconvoluted and used in the analysis. The deconvolution analysis of pupillary responses confirmed that the responses to beep sound were more likely to be modulated by the emotional state rather than being influenced by the short presentation interval between the images and sounds. These findings suggested that pupil size index modulations in the compelling situation between emotional and cognitive processing.

The effects of positive or negative self-talk on the alteration of brain functional connectivity by performing cognitive tasks

Self-talk can improve cognitive performance, but the underlying mechanism of such improvement has not been investigated. This study aimed to elucidate the effects of self-talks on functional connectivity associated with cognitive performance. We used the short form of Progressive Matrices Test (sRPM) to measure differences in performance improvements between self-respect and self-criticism. Participants were scanned using functional magnetic resonance imaging in the following order: baseline, during-sRPM1, post-sRPM1, self-respect or self-criticism, during-sRPM2, and post-sRPM2. Analysis was conducted to identify the self-talks' modulatory effects on the reward-motivation, default mode, and central-executive networks. Increase in sRPM2 score compared to sRPM1 score was observed only after self-criticism. The self-talk-by-repetition interaction effect was not found for during-sRPM, but found for post-sRPM; decreased nucleus accumbens-based connectivity was shown after self-criticism compared with self-respect. However, the significant correlations between the connectivity change and performance change appeared only in the self-respect group. Our findings showed that positive self-talk and negative self-talk differently modulate brain states concerning cognitive performance. Self-respect may have both positive and negative effects due to enhanced executive functions and inaccurate confidence, respectively, whereas self-criticism may positively affect cognitive performance by inducing a less confident state that increases internal motivation and attention.

Positive affective priming decreases the middle late positive potential response to negative images

INTRODUCTION

This study aims to expand on previous literature showing that incidental emotion state priming in a specific domain leads to a higher probability that the primed emotion domain will be activated during a subsequent task.

METHODS

To that end, we investigated the influence of happy, fearful, and neutral incidental emotion state priming on subsequent responses to emotionally negative and neutral pictures, measured by the event-related potential (ERP) late positive potential (LPP). New to our study, we examined the influence of affective priming on the LPP response (analyzed separately at early and middle latency ranges) to emotional pictures in both the foveal and extrafoveal presentation locations.

RESULTS

Following both fearful and neutral incidental state priming, both the early and middle LPP latency ranges overwhelmingly differentiated between negative and neutral pictures. Following happy incidental state priming, however, the LPP response failed to differentiate between negative and neutral pictures by the middle LPP latency range (800-1,000 ms). These results suggest that incidental happy states can have a protective effect when viewing aversive stimuli. Additionally, the LPP showed greater sensitivity to negative stimuli when presented extrafoveally compared to foveally.

CONCLUSIONS

Overall, our findings suggest that incidental affective state and stimulus location influence emotional processing differentially for emotionally negative and emotionally neutral stimuli.

Does emotion change auditory prediction and deviance detection?

In the last decades, a growing number of studies provided compelling evidence supporting the interplay of cognitive and affective processes. However, it remains to be clarified whether and how an emotional context affects the prediction and detection of change in unattended sensory events. In an event-related potential (ERP) study, we probed the modulatory role of pleasant, unpleasant and neutral visual contexts on the brain response to automatic detection of change in spectral (intensity) vs. temporal (duration) sound features. Twenty participants performed a passive auditory oddball task. Additionally, we tested the relationship between ERPs and self-reported mood. Participants reported more negative mood after the negative block. The P2 amplitude elicited by standards was increased in a positive context. Mismatch Negativity (MMN) amplitude was decreased in the negative relative to the neutral and positive contexts, and was associated with self-reported mood. These findings suggest that the detection of regularities in the auditory stream was facilitated in a positive context, whereas a negative visual context interfered with prediction error elicitation, through associated mood changes. Both ERP and behavioral effects highlight the intricate links between emotion, perception and cognitive processes.

Interhemispheric Asymmetries and Theta Activity in the Rostral Anterior Cingulate Cortex as EEG Signature of HIV-Related Depression: Gender Matters

Resting EEGs of 40 people living with HIV (PLWH) on long-term antiretroviral treatment were examined for z-scored deviations from a healthy control (normative database) to examine the main and interaction effects of depression and gender. Regions of interest were frontal (alpha) and central (all bands) for interhemispheric asymmetries in quantitative EEGs and theta in the rostral anterior cingulate cortex (rACC) in low-resolution electromagnetic tomography (LORETA). Z-scored normed deviations of depressed PLWH, compared with nondepressed, showed right-dominant interhemispheric asymmetries in all regions. However, after adjusting for multiple testing, significance remained only central for theta, alpha, and beta. Reversed (left-dominant) frontal alpha asymmetry is a potential EEG marker of depression in the HIV negative population that was not reversed in depressive PLWH; however, corresponding with extant literature, gender had an effect on the size of frontal alpha asymmetry. The LORETA analysis revealed a trending interactional effect of depression and gender on theta activity in the rACC in Brodmann area 32. We found that compared to men, women had greater right-dominant frontal alpha-asymmetry and elevated theta activity in voxels of the rACC, which may indicate less likelihood of depression and a higher likelihood of response to antidepressants. In conclusion, subtle EEG deviations, such as right-dominant central theta, alpha, and beta asymmetries and theta activity in the rACC may mark HIV-related depressive symptoms and may predict the likelihood of response to antidepressants but gender effects need to be taken into account. Although this study introduced the use of LORETA to examine the neurophysiological correlates of negative affect in PLWH, further research is needed to assess the utility of this tool in diagnostics and treatment monitoring of depression in PLWH.

Event-related lateralized readiness potential correlates of the emotion-priming Simon effect

The Simon effect indicates that the reaction time (RT) is shorter when the stimulus and response locations are congruent than when they are not. This study used a priming-target paradigm to explore the emotion-priming Simon effect with event-related potential techniques. The technique of residue iteration decomposition was employed to analyze the lateralized readiness potential (LRP) component, which contributed to disentangling the overlap between LRP and N2 central contralateral in the Simon task with horizontal stimulus-response arrangements. The behavioral result revealed significant Simon effect in RT. In the neural process, the Simon effect was reflected by both the stimulus-locked LRP (S-LRP) and the response-locked LRP (R-LRP), with the incongruent condition showing longer onset latency, larger Gratton-dip, and smaller negative-going deflection of S-LRP and smaller negative-going deflection of R-LRP. These findings suggest that the interference of irrelevant location information is located at the perceptual-encoding (indicated by S-LRP) and response-execution stages (indicated by R-LRP), providing evidence for both the perceptual-interference and response-interference accounts. However, the further linear regression result signaled that the Simon effect might be more closely related to the response-execution stage than the perceptual-encoding stage. In addition, the influence of emotion on the Simon effect was salient only in the incongruent condition, showing longer onset latency of S-LRP and larger Gratton-dip of R-LRP in the negative emotion-priming condition than in the neutral emotion-priming condition, which revealed that the emotional interference effect arose from the stages of perceptual encoding and early response execution only when the locations of a stimulus and the corresponding response were incongruent.

Modulation of auditory spatial attention by visual emotional cues: differential effects of attentional engagement and disengagement for pleasant and unpleasant cues

Previous research has demonstrated that threatening, compared to neutral pictures, can bias attention towards non-emotional auditory targets. Here we investigated which subcomponents of attention contributed to the influence of emotional visual stimuli on auditory spatial attention. Participants indicated the location of an auditory target, after brief (250 ms) presentation of a spatially non-predictive peripheral visual cue. Responses to targets were faster at the location of the preceding visual cue, compared to at the opposite location (cue validity effect). The cue validity effect was larger for targets following pleasant and unpleasant cues compared to neutral cues, for right-sided targets. For unpleasant cues, the crossmodal cue validity effect was driven by delayed attentional disengagement, and for pleasant cues, it was driven by enhanced engagement. We conclude that both pleasant and unpleasant visual cues influence the distribution of attention across modalities and that the associated attentional mechanisms depend on the valence of the visual cue.

Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains

In everyday life, multiple sensory channels jointly trigger emotional experiences and one channel may alter processing in another channel. For example, seeing an emotional facial expression and hearing the voice’s emotional tone will jointly create the emotional experience. This example, where auditory and visual input is related to social communication, has gained considerable attention by researchers. However, interactions of visual and auditory emotional information are not limited to social communication but can extend to much broader contexts including human, animal, and environmental cues. In this article, we review current research on audiovisual emotion processing beyond face-voice stimuli to develop a broader perspective on multimodal interactions in emotion processing. We argue that current concepts of multimodality should be extended in considering an ecologically valid variety of stimuli in audiovisual emotion processing. Therefore, we provide an overview of studies in which emotional sounds and interactions with complex pictures of scenes were investigated. In addition to behavioral studies, we focus on neuroimaging, electro- and peripher-physiological findings. Furthermore, we integrate these findings and identify similarities or differences. We conclude with suggestions for future research.

The influence of negative emotion on the Simon effect as reflected by p300

The Simon effect refers to the phenomenon that reaction time (RT) is faster when stimulus and response location are congruent than when they are not. This study used the priming-target paradigm to explore the influence of induced negative emotion on the Simon effect with event-related potential techniques (ERPs). The priming stimuli were composed of two kinds of pictures, the negative and neutral pictures, selected from the International Affective Picture System (IAPS). The target stimuli included chessboards of two color types. One was red and black the other one was green and black. Each chessboard was presented on the left or the right of the screen. The participants were asked to press the response keys according to the colors of the chessboards. It was called the congruent condition if the chessboard and the response key were on the same side, otherwise incongruent condition. In this study, the emotion-priming Simon effect was found in terms of RT and P300. Negative emotion compared with neutral emotion significantly enhanced the Simon effect in the cognitive process, reflected by a larger difference of P300 latency between the incongruent and congruent trials. The results suggest that the induced negative emotion influenced the Simon effect at the late stage of the cognitive process, and the P300 latency could be considered as the reference measure. These findings may be beneficial to researches in psychology and industrial engineering in the future.

HIV-positive females show blunted neurophysiological responses in an emotion-attention dual task paradigm

OBJECTIVE

Although HIV is associated with decreased emotional and cognitive functioning, the mechanisms through which affective changes can alter cognitive processes in HIV-infected individuals are unknown. We aimed to clarify this question through testing the extent to which emotionally negative stimuli prime attention to a subsequent infrequently occurring auditory tone in HIV+ compared to HIV- females.

METHODS

Attention to emotional compared to non-emotional pictures was measured via the LPP ERP. Subsequent attention was indexed through the N1 and late processing negativity ERP. We also assessed mood and cognitive functioning in both groups.

RESULTS

In HIV- females, emotionally negative pictures, compared to neutral pictures, resulted in an enhanced LPP to the pictures and an enhanced N1 to subsequent tones. The HIV+ group did not show a difference in the LPP measure between picture categories, and accordingly, did not show a priming effect to the subsequent infrequent tones.

CONCLUSIONS

The ERP findings, combined with neuropsychological deficits, suggest that HIV+ females show impairments in attention to emotionally-laden stimuli and that this impairment might be related to a loss of affective priming.

SIGNIFICANCE

This study is the first to provide physiological evidence that the LPP, a measure of attention to emotionally-charged visual stimuli, is reduced in HIV-infected individuals. These results set the stage for future work aimed at localizing brain activation to emotional stimuli in HIV+ individuals.

Phasic boosting of auditory perception by visual emotion

Emotionally negative stimuli boost perceptual processes. There is little known, however, about the timing of this modulation. The present study aims at elucidating the phasic effects of, emotional processing on auditory processing within subsequent time-windows of visual emotional, processing in humans. We recorded the electroencephalogram (EEG) while participants responded to a, discrimination task of faces with neutral or fearful expressions. A brief complex tone, which subjects, were instructed to ignore, was displayed concomitantly, but with different asynchronies respective to, the image onset. Analyses of the N1 auditory event-related potential (ERP) revealed enhanced brain, responses in presence of fearful faces. Importantly, this effect occurred at picture-tone asynchronies of, 100 and 150ms, but not when these were displayed simultaneously, or at 50ms or 200ms asynchrony. These results confirm the existence of a fast-operating crossmodal effect of visual emotion on auditory, processing, suggesting a phasic variation according to the time-course of emotional processing.

Delayed attentional disengagement from sad face: a study of alpha rhythm by event-related desynchronization

This study investigated the influence of different emotional facial cues on the detection of subsequent visual digit targets presented after various cue-target intervals (CTIs). Behavioral results indicated that, compared to neutral faces, happy faces facilitated the response to subsequent tasks only after a short CTI (17 ms), while sad faces would slow or inhibit the processing of following tasks after different CTIs (17, 350, 1000, and 1500 ms). Event-related desynchronization of alpha rhythm (α-ERD) showed that the left frontal and parietal cortical areas were more prominently activated by emotional faces than by neutral ones. In particular, happy faces induced more activity in left frontal lobes, starting from the beginning of CTI (post-cue 0~400 ms), while sad faces induced stronger and longer activation during the middle of CTI (post-cue 400~800 ms). Such a later a-ERD in left frontal area suggested that the attentional disengagement was slowed by sad faces.