ERP evidence on the interaction between information structure and emotional salience of words

A premium for positive social interest and attractive voices in the acceptability of unfair offers? An ERP study

Although the attractiveness of voices plays an important role in social interactions, it is unclear how voice attractiveness and social interest influence social decision-making. Here, we combined the ultimatum game with recording event-related brain potentials (ERPs) and examined the effect of attractive versus unattractive voices of the proposers, expressing positive versus negative social interest ("I like you" vs. "I don't like you"), on the acceptance of the proposal. Overall, fair offers were accepted at significantly higher rates than unfair offers, and high voice attractiveness increased acceptance rates for all proposals. In ERPs in response to the voices, their attractiveness and expressed social interests yielded early additive effects in the N1 component, followed by interactions in the subsequent P2, P3 and N400 components. More importantly, unfair offers elicited a larger Medial Frontal Negativity (MFN) than fair offers but only when the proposer's voice was unattractive or when the voice carried positive social interest. These results suggest that both voice attractiveness and social interest moderate social decision-making and there is a similar "beauty premium" for voices as for faces.

An EPR study of the cognitive processes underlying the impact of self-relevant information on emotional word processing

Using the event-related potentials (ERPs) technique, this study successively presented names (in either a supra- or subthreshold manner) and emotional words to examine how self-relevant cue (self-name) affects emotional word processing in word class judgment task (to determine whether an emotional word is a noun or adjective) and valence judgment task (to determine whether an emotional word is positive or negative). At the suprathreshold condition, self-relevant positive words elicited a more significant Early posterior negativity (EPN) than negative words only in the valence judgment task. In contrast, at the subthreshold condition, self-relevant negative words elicited an enhanced Late positive potential (LPP) than positive words only in the word class judgment task. These results indicate that self-relevant cue affects emotional word processing at both suprathreshold and subthreshold conditions; nevertheless, the effect manifests as self-positive bias at the suprathreshold condition and self-negative bias at the subthreshold condition. The experimental task modulates these dynamics.

The early negative bias of social semantics: evidence from behavioral and ERP studies

BACKGROUND

Compared to nonsocial information, the human brain is more highly sensitive to social information. As a kind of typical social semantic information, the words describing person traits differ from the nonsocial semantic information describing inanimate objects in many ways. It remains to be seen whether the processing of trait words has a valence asymmetric and whether it differs from the processing of nonsocial semantic information in terms of behavioral responses and neural temporal processes.

METHOD

Taking person and object names as priming stimuli and adjective words only used for describing humans or objects as target stimuli, the present study aimed to investigate the processing characteristics of social and nonsocial semantic information by recording both behavioral and ERP data.

RESULTS

Behavioral results showed that the response times for negative words were significantly slower than those for positive words whether for social or nonsocial semantic information. The accuracy rates of negative words were significantly lower than those of positive words when the targets were social words which is contrary to the nonsocial words. The ERP results indicated that there was a negative bias effect on the processing of both types of information during the whole time course of brain neural activity; that is, the P2, N400, and LPP amplitudes elicited by negative words were larger than those elicited by positive words; However, the negative bias effect of social semantic information started at the early perceptual stage which was significantly earlier than the onset of negative bias of nonsocial semantic information, and was significantly affected by the prime type. In addition, there was a significant semantic conflict N400 effect only for nonsocial semantic information.

CONCLUSIONS

Overall, the present study revealed the existence of an early negative bias of social information and provided evidence for the specificity of social information.

ERP evidence for emotion-specific congruency effects between sentences and new words with disgust and sadness connotations

Introduction

The present study investigated how new words with acquired connotations of disgust and sadness, both negatively valenced but distinctive emotions, modulate the brain dynamics in the context of emotional sentences.

Methods

Participants completed a learning session in which pseudowords were repeatedly paired with faces expressing disgust and sadness. An event-related potential (ERP) session followed the next day, in which participants received the learned pseudowords (herein, new words) combined with sentences and were asked to make emotional congruency judgment.

Results

Sad new words elicited larger negative waveform than disgusting new words in the 146-228 ms time window, and emotionally congruent trials showed larger positive waveform than emotionally incongruent trials in the 304-462 ms time window. Moreover, the source localization in the latter suggested that congruent trials elicited larger current densities than incongruent trials in a number of emotion-related brain structures (e.g., the orbitofrontal cortex and cingulate gyrus) and language-related brain structures (e.g., the temporal lobe and the lingual gyrus).

Discussion

These results suggested that faces are an effective source for the acquisition of words' emotional connotations, and such acquired connotations can generate semantic and emotional congruency effects in sentential contexts.

Emotion recognition support system: Where physicians and psychiatrists meet linguists and data engineers

An important factor in the course of daily medical diagnosis and treatment is understanding patients’ emotional states by the caregiver physicians. However, patients usually avoid speaking out their emotions when expressing their somatic symptoms and complaints to their non-psychiatrist doctor. On the other hand, clinicians usually lack the required expertise (or time) and have a deficit in mining various verbal and non-verbal emotional signals of the patients. As a result, in many cases, there is an emotion recognition barrier between the clinician and the patients making all patients seem the same except for their different somatic symptoms. In particular, we aim to identify and combine three major disciplines (psychology, linguistics, and data science) approaches for detecting emotions from verbal communication and propose an integrated solution for emotion recognition support. Such a platform may give emotional guides and indices to the clinician based on verbal communication at the consultation time.

How accentuation influences the processing of emotional words in spoken language: An ERP study

Pitch accent marks information structure in utterances in many languages but little is known about the effects of accent on the perception of emotional word meaning. The present study explored the processing of accentuation and its influence on the semantic integration of emotional words during spoken sentence comprehension. Twenty-five participants were presented with sets of spoken Chinese sentences while accentuation and emotional meaning of the adjectives were orthogonally manipulated. An implicit task required the recognition of words contained in the sentences, whereas an explicit task required judging the presence of an accented or emotional word. In the ERPs to the adjectives, accentuation induced a long-lasting anterior negativity starting around 150-250 ms and a late posterior positivity. More importantly, emotionally negative words elicited larger negativities between 300 and 700 ms as compared to neutral words but only when they were accented. Interestingly, these negativities showed a parietal N400-typical distribution when accent was implicit but strongly overlapped with the accent-induced anterior negativity when accent was task-relevant. Hence, accentuation may enable the processing of emotional meaning by directing attention towards the accented words. When accent and emotion are explicit parts of the task, similar frontal attentional networks are activated by emotion as by accent alone. In contrast, when accent and emotion are implicit to the task, emotion appears to merely activate parietal networks, typical for semantic integration effects. Together, these results suggest that accentuation plays an important role in spoken sentence comprehension, deserving further study.

ERP evidence of age-related differences in emotional processing

The present study used event-related potentials (ERPs) to examine differences in the temporal dynamics of emotion processing in young and older adults, with a specific focus on the positivity effect, that is, the preferential processing of positive over negative information. To this aim, we used a language paradigm that allowed us to investigate early ERP components as well as later components, namely the N400 and the late positive complex (LPC). Young and older adults were presented with neutral sentence stems with positive, negative or neutral/semantically-incongruent critical word endings while their electrical brain activity was recorded. There were no effects of emotional valence on early ERP components. Instead, a positivity effect was evident in young adults indexed by reduced N400s for positive sentence endings. Perhaps due to reduced semantic processing abilities, older adults did not show any N400 effect. ERP effects in this group were evident at a later processing stage and took the form of larger LPCs for neutral/incongruent information. Overall, there was no effect of emotional valence on either the N400 or the LPC in older adults. Our data suggest that with age, more effortful semantic processing may deplete resources for emotional processing.

The N400/FN400 and Lateralized Readiness Potential Neural Correlates of Valence and Origin of Words' Affective Connotations in Ambiguous Task Processing

Recent behavioral studies revealed an interesting phenomenon concerning the influence of affect on the interpretation of ambiguous stimuli. In a paradigm, where the participants' task was to read a word, remember its meaning for a while, and then choose one of two pictorial-alphabet-like graphical signs best representing the word sense, we observed that the decisions involving trials with reflective-originated verbal stimuli were performed significantly longer than decisions concerning other stimuli (i.e., automatic-originated). The origin of an affective reaction is a dimension which allows speaking of an affect as automatic (you feel it in your guts) or reflective (you feel it comes from your mind). The automatic affective reaction represents the immediate and inescapable as opposed to the reflective, i.e., the delayed and controllable affective responses to stimuli. In the current experiment, we investigated the neural correlates of performance in an QR-signs-selection ambiguous task. We found the effects of valence and origin in the N400/FN400 potential by means of a stimuli-locked analysis of the initial part of the task, that is, the remembering of a certain word stimulus in a working memory. The N400/FN400 effects were separated in space on scalp distribution. Reflective originated stimuli elicited more negative FN400 than other conditions, which means that such stimuli indeed are associated with conceptual incongruence or higher affective complexity of meaning, but distinct from purely cognitive concreteness. Moreover, the amplitude of the potential preceding the decision, analyzed in the response-locked way, was shaped by the origin of an affective state but not valence. Trials involving decisions concerning reflective-originated words were characterized by a more negative amplitude than trials involving automatic-originated and control word stimuli. This corresponds to the observed pattern of response latencies, where we found that latencies for reflective stimuli were longer than for automatic originated or control ones. Additionally, this study demonstrates that the proposed new ambiguous paradigm is useful in studies concerning the influence of affect on decisions.

Concurrent emotional response and semantic unification: An event-related potential study

Using event-related potentials, in this study we examined how implied emotion is derived from sentences. In the same sentential context, different emotionally neutral words rendered the whole sentence emotionally neutral and semantically congruent, emotionally negative and semantically congruent, or emotionally neutral and semantically incongruent. Relative to the words in the neutral-congruent condition, the words in the neutral-incongruent condition elicited a larger N400, indicating increased semantic processing, whereas the words in the negative-congruent condition elicited a long-lasting positivity between 300 and 1,000 ms, indicating an emotional response. The overlapping time windows of semantic processing and the emotional response suggest that the construction of emotional meaning operates concurrently with semantic unification. The results indicate that the implied emotional processing of sentences may be a result of unification operations but does not necessarily involve causal appraisal of a sentence's mental representation.

The Interplay between Topic Shift and Focus in the Dynamic Construction of Discourse Representations

Previous studies have suggested that focusing an element can enhance the activation of the focused element and bring about a number of processing benefits. However, whether and how this local prominence of information interacts with global discourse organization remains unclear. In the present study, we addressed this issue in two experiments. Readers were presented with four-sentence discourses. The first sentence of each discourse contained a critical word that was either focused or unfocused in relation to a wh-question preceding the discourse. The second sentence either maintained or shifted the topic of the first sentence. Participants were told to read for comprehension and for a probe recognition task in which the memory of the critical words was tested. In Experiment 1, when the probe words were tested immediately after the point of topic shift, we found shorter response times for the focused critical words than the unfocused ones regardless of topic manipulation. However, in Experiment 2, when the probe words were tested two sentences away from the point of topic shift, we found the facilitation effect of focus only in the topic-maintained discourses, but not in the topic-shifted discourses. This suggests that the facilitation effect of focus was not immediately suppressed at the point of topic shifting, but when additional information was added to the new topic. Our findings provide evidence for the dynamic interplay between global topic structure and local salience of information and have important implications on how activation of information fluctuates in mental representation.

Vivid: How valence and arousal influence word processing under different task demands

In this study, we used event-related potentials to examine how different dimensions of emotion-valence and arousal-influence different stages of word processing under different task demands. In two experiments, two groups of participants viewed the same single emotional and neutral words while carrying out different tasks. In both experiments, valence (pleasant, unpleasant, and neutral) was fully crossed with arousal (high and low). We found that the task made a substantial contribution to how valence and arousal modulated the late positive complex (LPC), which is thought to reflect sustained evaluative processing (particularly of emotional stimuli). When participants performed a semantic categorization task in which emotion was not directly relevant to task performance, the LPC showed a larger amplitude for high-arousal than for low-arousal words, but no effect of valence. In contrast, when participants performed an overt valence categorization task, the LPC showed a large effect of valence (with unpleasant words eliciting the largest positivity), but no effect of arousal. These data show not only that valence and arousal act independently to influence word processing, but that their relative contributions to prolonged evaluative neural processes are strongly influenced by the situational demands (and by individual differences, as revealed in a subsequent analysis of subjective judgments).

Understanding approach and avoidance in verbal descriptions of everyday actions: An ERP study

Understanding verbal descriptions of everyday actions could involve the neural representation of action direction (avoidance and approach) toward persons and things. We recorded the electrophysiological activity of participants while they were reading approach/avoidance action sentences that were directed toward a target: a thing/a person (i.e., "Petra accepted/rejected Ramón in her group"/ "Petra accepted/rejected the receipt of the bank"). We measured brain potentials time locked to the target word. In the case of things, we found a N400-like component with right frontal distribution modulated by approach/avoidance action. This component was more negative in avoidance than in approach sentences. In the case of persons, a later negative event-related potential (545-750 ms) with left frontal distribution was sensitive to verb direction, showing more negative amplitude for approach than avoidance actions. In addition, more negativity in approach-person sentences was associated with fear avoidance trait, whereas less negativity in avoidance-person sentences was associated with a greater approach trait. Our results support that verbal descriptions of approach/avoidance actions are encoded differently depending on whether the target is a thing or a person. Implications of these results for a social, emotional and motivational understanding of action language are discussed.

Dynamic Effects of Self-Relevance and Task on the Neural Processing of Emotional Words in Context

We used event-related potentials (ERPs) to examine the interactions between task, emotion, and contextual self-relevance on processing words in social vignettes. Participants read scenarios that were in either third person (other-relevant) or second person (self-relevant) and we recorded ERPs to a neutral, pleasant, or unpleasant critical word. In a previously reported study (Fields and Kuperberg, 2012) with these stimuli, participants were tasked with producing a third sentence continuing the scenario. We observed a larger LPC to emotional words than neutral words in both the self-relevant and other-relevant scenarios, but this effect was smaller in the self-relevant scenarios because the LPC was larger on the neutral words (i.e., a larger LPC to self-relevant than other-relevant neutral words). In the present work, participants simply answered comprehension questions that did not refer to the emotional aspects of the scenario. Here we observed quite a different pattern of interaction between self-relevance and emotion: the LPC was larger to emotional vs. neutral words in the self-relevant scenarios only, and there was no effect of self-relevance on neutral words. Taken together, these findings suggest that the LPC reflects a dynamic interaction between specific task demands, the emotional properties of a stimulus, and contextual self-relevance. We conclude by discussing implications and future directions for a functional theory of the emotional LPC.

The emotion potential of simple sentences: additive or interactive effects of nouns and adjectives?

The vast majority of studies on affective processes in reading focus on single words. The most robust finding is a processing advantage for positively valenced words, which has been replicated in the rare studies investigating effects of affective features of words during sentence or story comprehension. Here we were interested in how the different valences of words in a sentence influence its processing and supralexical affective evaluation. Using a sentence verification task we investigated how comprehension of simple declarative sentences containing a noun and an adjective depends on the valences of both words. The results are in line with the assumed general processing advantage for positive words. We also observed a clear interaction effect, as can be expected from the affective priming literature: sentences with emotionally congruent words (e.g., The grandpa is clever) were verified faster than sentences containing emotionally incongruent words (e.g., The grandpa is lonely). The priming effect was most prominent for sentences with positive words suggesting that both, early processing as well as later meaning integration and situation model construction, is modulated by affective processing. In a second rating task we investigated how the emotion potential of supralexical units depends on word valence. The simplest hypothesis predicts that the supralexical affective structure is a linear combination of the valences of the nouns and adjectives (Bestgen, 1994). Overall, our results do not support this: The observed clear interaction effect on ratings indicate that especially negative adjectives dominated supralexical evaluation, i.e., a sort of negativity bias in sentence evaluation. Future models of sentence processing thus should take interactive affective effects into account.

The emotion potential of words and passages in reading Harry Potter--an fMRI study

Previous studies suggested that the emotional connotation of single words automatically recruits attention. We investigated the potential of words to induce emotional engagement when reading texts. In an fMRI experiment, we presented 120 text passages from the Harry Potter book series. Results showed significant correlations between affective word (lexical) ratings and passage ratings. Furthermore, affective lexical ratings correlated with activity in regions associated with emotion, situation model building, multi-modal semantic integration, and Theory of Mind. We distinguished differential influences of affective lexical, inter-lexical, and supra-lexical variables: differential effects of lexical valence were significant in the left amygdala, while effects of arousal-span (the dynamic range of arousal across a passage) were significant in the left amygdala and insula. However, we found no differential effect of passage ratings in emotion-associated regions. Our results support the hypothesis that the emotion potential of short texts can be predicted by lexical and inter-lexical affective variables.

Low social status decreases the neural salience of unfairness

Social hierarchy exists in almost all social species and affects everything from resource allocation to the development of intelligence. Previous studies showed that status within a social hierarchy influences the perceived fairness of income allocation. However, the effect of one’s social status on economic decisions is far from clear, as are the neural processes underlying these decisions. In this study, we dynamically manipulated participants’ social status and analyzed their behavior as recipients in the ultimatum game (UG), during which event-related potentials (ERPs) were recorded. Behavioral results showed that acceptance rates for offers increased with the fairness level of offers. Importantly, participants were less likely to accept unfair offers when they were endowed with high status than with low status. In addition, cues indicating low status elicited a more positive P2 than cues indicating high status in an earlier time window (170–240 ms), and cues indicating high status elicited a more negative N400 than cues indicating low status in a later time window (350–520 ms). During the actual reception of offers, the late positivity potential (LPP, 400–700 ms) for unfair offers was more positive in the high status condition than in the low status condition, suggesting a decreased arousal for unfair offers during low status. These findings suggest a strong role of social status in modulating individual behavioral and neural responses to fairness.

The dynamic influence of emotional words on sentence processing

In the present study, we aimed to examine how the emotionality of words influences online sentence processing-specifically, the influence of emotional words on the processing of following words in sentences. We manipulated the emotionality of verbs as well as the orthographic correctness of their following (neutral) object nouns, so that the orthographic violation of the (neutral) nouns occurred in either emotional or neutral sentences. Event-related potentials (ERPs) were recorded to both the nouns and the verbs. We found that the orthographic violation of the nouns elicited a P2 and an N400 effect in the emotionally neutral sentences, but an LPC effect in the emotionally charged sentences. We also found that the emotional verbs elicited a larger N1, a larger P2, and a larger N400 than did the neutral verbs. The ERP results suggest that emotional words capture more attention than neutral words, which further affects early orthographic analysis of the following words. Our findings demonstrate a dynamic influence of emotional words on sentence processing.

The building blocks of social communication

In the present review, social communication will be discussed in the context of social cognition, and cold and hot cognition. The review presents research on prosody, processing of faces, multimodal processing of voice and face, and the impact of emotion on constructing semantic meaning. Since the focus of this mini review is on brain processes involved in these cognitive functions, the bulk of evidence presented will be from event related potential (ERP) studies as this methodology offers the best temporal resolution of cognitive events under study. The argument is made that social communication is accomplished via fast acting sensory processes and later, top down processes. Future directions both in terms of methodology and research questions are also discussed.