Electrophysiological evidence of facial inversion with rapid serial visual presentation

The dissociation of semantically congruent and incongruent cross-modal effects on the visual attentional blink

Introduction

Recent studies have found that the sound-induced alleviation of visual attentional blink, a well-known phenomenon exemplifying the beneficial influence of multisensory integration on time-based attention, was larger when that sound was semantically congruent relative to incongruent with the second visual target (T2). Although such an audiovisual congruency effect has been attributed mainly to the semantic conflict carried by the incongruent sound restraining that sound from facilitating T2 processing, it is still unclear whether the integrated semantic information carried by the congruent sound benefits T2 processing.

Methods

To dissociate the congruence-induced benefit and incongruence-induced reduction in the alleviation of visual attentional blink at the behavioral and neural levels, the present study combined behavioral measures and event-related potential (ERP) recordings in a visual attentional blink task wherein the T2-accompanying sound, when delivered, could be semantically neutral in addition to congruent or incongruent with respect to T2.

Results

The behavioral data clearly showed that compared to the neutral sound, the congruent sound improved T2 discrimination during the blink to a higher degree while the incongruent sound improved it to a lesser degree. The T2-locked ERP data revealed that the early occipital cross-modal N195 component (192-228 ms after T2 onset) was uniquely larger in the congruent-sound condition than in the neutral-sound and incongruent-sound conditions, whereas the late parietal cross-modal N440 component (400-500 ms) was prominent only in the incongruent-sound condition.

Discussion

These findings provide strong evidence that the modulating effect of audiovisual semantic congruency on the sound-induced alleviation of visual attentional blink contains not only a late incongruence-induced cost but also an early congruence-induced benefit, thereby demonstrating for the first time an unequivocal congruent-sound-induced benefit in alleviating the limitation of time-based visual attention.

Impact of observational and direct learning on fear conditioning generalization in humans

Humans gain knowledge about threats not only from their own experiences but also from observing others' behavior. A neutral stimulus is associated with a threat stimulus for several times and the neutral stimulus will evoke fear responses, which is known as fear conditioning. When encountering a new event that is similar to one previously associated with a threat, one may feel afraid and produce fear responses. This is called fear generalization. Previous studies have mostly focused on fear conditioning and generalization based on direct learning, but few have explored how observational fear learning affects fear conditioning and generalization. To the best of our knowledge, no previous study has focused on the neural correlations of fear conditioning and generalization based on observational learning. In the present study, 58 participants performed a differential conditioning paradigm in which they learned the associations between neutral cues (i.e., geometric figures) and threat stimuli (i.e., electric shock). The learning occurred on their own (i.e., direct learning) and by observing other participant's responses (i.e., observational learning); the study used a within-subjects design. After each learning condition, a fear generalization paradigm was conducted by each participant independently while their behavioral responses (i.e., expectation of a shock) and electroencephalography (EEG) recordings or responses were recorded. The shock expectancy ratings showed that observational learning, compared to direct learning, reduced the differentiation between the conditioned threatening stimuli and safety stimuli and the increased shock expectancy to the generalization stimuli. The EEG indicated that in fear learning, threatening conditioned stimuli in observational and direct learning increased early discrimination (P1) and late motivated attention (late positive potential [LPP]), compared with safety conditioned stimuli. In fear generalization, early discrimination, late motivated attention, and orienting attention (alpha-event-related desynchronization [alpha-ERD]) to generalization stimuli were reduced in the observational learning condition. These findings suggest that compared to direct learning, observational learning reduces differential fear learning and increases the generalization of fear, and this might be associated with reduced discrimination and attentional function related to generalization stimuli.

The interplay between audiovisual temporal synchrony and semantic congruency in the cross-modal boost of the visual target discrimination during the attentional blink

The visual attentional blink can be substantially reduced by delivering a task-irrelevant sound synchronously with the second visual target (T2), and this effect is further modulated by the semantic congruency between the sound and T2. However, whether the cross-modal benefit originates from audiovisual interactions or sound-induced alertness remains controversial, and whether the semantic congruency effect is contingent on audiovisual temporal synchrony needs further investigation. The current study investigated these questions by recording event-related potentials (ERPs) in a visual attentional blink task wherein a sound could either synchronize with T2, precede T2 by 200 ms, be delayed by 100 ms, or be absent, and could be either semantically congruent or incongruent with T2 when delivered. The behavioral data showed that both the cross-modal boost of T2 discrimination and the further semantic modulation were the largest when the sound synchronized with T2. In parallel, the ERP data yielded that both the early occipital cross-modal P195 component (192-228 ms after T2 onset) and late parietal cross-modal N440 component (424-448 ms) were prominent only when the sound synchronized with T2, with the former being elicited solely when the sound was further semantically congruent whereas the latter occurring only when that sound was incongruent. These findings demonstrate not only that the cross-modal boost of T2 discrimination during the attentional blink stems from early audiovisual interactions and the semantic congruency effect depends on audiovisual temporal synchrony, but also that the semantic modulation can unfold at the early stage of visual discrimination processing.

Nudging the N170 forward with prior stimulation-Bridging the gap between N170 and recognition potential

Evoked response potentials are often divided up into numerous components, each with their own body of literature. But is there less variety than we might suppose? In this study, we nudge one component into looking like another. Both the N170 and recognition potential (RP) are N1 components in response to familiar objects. However, the RP is often measured with a forward mask that ends at stimulus onset whereas the N170 is often measured with no masking at all. This study investigates how inter‐stimulus interval (ISI) may delay and distort the N170 into an RP by manipulating the temporal gap (ISI) between forward mask and target. The results revealed reverse relationships between the ISI on the one hand, and the N170 latency, single‐trial N1 jitter (an approximation of N1 width) and reaction time on the other hand. Importantly, we find that scalp topographies have a unique signature at the N1 peak across all conditions, from the longest gap (N170) to the shortest (RP). These findings prove that the mask‐delayed N1 is still the same N170, even under conditions that are normally associated with a different component like the RP. In general, our results suggest greater synthesis in the study of event related potential components.

Attentional blink suppresses both stimulus-driven and representation-driven cross-modal spread of attention

Previous studies have shown that visual attention effect can spread to the task-irrelevant auditory modality automatically through either the stimulus-driven binding process or the representation-driven priming process. Using an attentional blink paradigm, the present study investigated whether the long-latency stimulus-driven and representation-driven cross-modal spread of attention would be inhibited or facilitated when the attentional resources operating at the post-perceptual stage of processing are inadequate, whereas ensuring all visual stimuli were spatially attended and the representations of visual target object categories were activated, which were previously thought to be the only endogenous prerequisites for triggering cross-modal spread of attention. The results demonstrated that both types of attentional spreading were completely suppressed during the attentional blink interval but were highly prominent outside the attentional blink interval, with the stimulus-driven process being independent of, whereas the representation-driven process being dependent on, audiovisual semantic congruency. These findings provide the first evidence that the occurrences of both stimulus-driven and representation-driven spread of attention are contingent on the amount of post-perceptual attentional resources responsible for the late consolidation processing of visual stimuli, whereas the early detection of visual stimuli and the top-down activation of the visual representations are not the sole endogenous prerequisites for triggering any types of cross-modal attentional spreading.

Neural Basis of Semantically Dependent and Independent Cross-Modal Boosts on the Attentional Blink

The present study recorded event-related potentials (ERPs) in a visual object-recognition task under the attentional blink paradigm to explore the temporal dynamics of the cross-modal boost on attentional blink and whether this auditory benefit would be modulated by semantic congruency between T2 and the simultaneous sound. Behaviorally, the present study showed that not only a semantically congruent but also a semantically incongruent sound improved T2 discrimination during the attentional blink interval, whereas the enhancement was larger for the congruent sound. The ERP results revealed that the behavioral improvements induced by both the semantically congruent and incongruent sounds were closely associated with an early cross-modal interaction on the occipital N195 (192–228 ms). In contrast, the lower T2 accuracy for the incongruent than congruent condition was accompanied by a larger late occurring cento-parietal N440 (424–448 ms). These findings suggest that the cross-modal boost on attentional blink is hierarchical: the task-irrelevant but simultaneous sound, irrespective of its semantic relevance, firstly enables T2 to escape the attentional blink via cross-modally strengthening the early stage of visual object-recognition processing, whereas the semantic conflict of the sound begins to interfere with visual awareness only at a later stage when the representation of visual object is extracted.

Neural Correlates of Emotion Processing in Word Detection Task

In our previous study, we have proposed a three-stage model of emotion processing; in the current study, we investigated whether the ERP component may be different when the emotional content of stimuli is task-irrelevant. In this study, a dual-target rapid serial visual presentation (RSVP) task was used to investigate how the emotional content of words modulates the time course of neural dynamics. Participants performed the task in which affectively positive, negative, and neutral adjectives were rapidly presented while event-related potentials (ERPs) were recorded from 18 undergraduates. The N170 component was enhanced for negative words relative to positive and neutral words. This indicates that automatic processing of negative information occurred at an early perceptual processing stage. In addition, later brain potentials such as the late positive potential (LPP) were only enhanced for positive words in the 480-580-ms post-stimulus window, while a relatively large amplitude signal was elicited by positive and negative words between 580 and 680 ms. These results indicate that different types of emotional content are processed distinctly at different time windows of the LPP, which is in contrast with the results of studies on task-relevant emotional processing. More generally, these findings suggest that a negativity bias to negative words remains to be found in emotion-irrelevant tasks, and that the LPP component reflects dynamic separation of emotion valence.

The Idea Is Good, but…: Failure to Replicate Associations of Oxytocinergic Polymorphisms with Face-Inversion in the N170

BACKGROUND

In event-related potentials, the N170 manifests itself especially in reaction to faces. In the healthy population, face-inversion leads to stronger negative amplitudes and prolonged latencies of the N170, effects not being present in patients with autism-spectrum-disorder (ASD). ASD has frequently been associated with differences in oxytocinergic neurotransmission. This ERP-study aimed to investigate the face-inversion effect in association with oxytocinergic candidate genes. It was expected that risk-allele-carriers of the oxytocin-receptor-gene-polymorphism (rs53576) and of CD38 (rs379863) responded similar to upright and inverted faces as persons with ASD. Additionally, reactions to different facial emotional expressions were studied. As there have been difficulties with replications of those molecular genetic association studies, we aimed to replicate our findings in a second study.

METHOD

Seventy-two male subjects in the first-, and seventy-eight young male subjects in the replication-study conducted a face-inversion-paradigm, while recording EEG. DNA was extracted from buccal cells.

RESULTS

Results revealed stronger N170-amplitudes and longer latencies in reaction to inverted faces in comparison to upright ones. Furthermore, effects of emotion on N170 were evident. Those effects were present in the first and in the second study. Whereas we found molecular-genetic associations of oxytocinergic polymorphisms with the N170 in the first study, we failed to do so in the replication sample.

CONCLUSION

Results indicate that a deeper theoretical understanding of this research-field is needed, in order to generate possible explanations for these findings. Results, furthermore, support the hypotheses that success of reproducibility is correlated with strength of lower original p-values and larger effect sizes in the original study.

Emotional noun processing: an ERP study with rapid serial visual presentation

Reading is an important part of our daily life, and rapid responses to emotional words have received a great deal of research interest. Our study employed rapid serial visual presentation to detect the time course of emotional noun processing using event-related potentials. We performed a dual-task experiment, where subjects were required to judge whether a given number was odd or even, and the category into which each emotional noun fit. In terms of P1, we found that there was no negativity bias for emotional nouns. However, emotional nouns elicited larger amplitudes in the N170 component in the left hemisphere than did neutral nouns. This finding indicated that in later processing stages, emotional words can be discriminated from neutral words. Furthermore, positive, negative, and neutral words were different from each other in the late positive complex, indicating that in the third stage, even different emotions can be discerned. Thus, our results indicate that in a three-stage model the latter two stages are more stable and universal.

Configural and featural face processing are differently modulated by attentional resources at early stages: an event-related potential study with rapid serial visual presentation

It is widely reported that face recognition relies on two dissociable mechanisms, the featural and the configural processing. However, it is unclear whether these two processing types involve different neural mechanisms and are differently modulated by attentional resources. Using the attentional blink (AB) paradigm, we aimed to investigate the effect of attentional resources on configural and featural face processing by recording event-related potentials (ERPs). The amount of attentional resources was manipulated as deficient or sufficient by presenting the second target (T2) in or out of the AB period, respectively. We found that in addition to a traditional P3 attention effect, the amplitude of N170/VPP to the T2 stimuli was also sensitive to attentional resources, suggesting that attention affects face processing at an earlier perceptual processing stage. More importantly, configural face processing elicited a larger posterior P1 compared to featural face processing, but only when the attentional resources were sufficient. In contrast, the anterior N1 was larger for configural relative to featural face processing only when the attentional resources were deficient. These results suggest that early stages of configural and featural face processing are differently modulated by attentional resources, possibly with different underlying mechanisms.

Three stages of emotional word processing: an ERP study with rapid serial visual presentation

Rapid responses to emotional words play a crucial role in social communication. This study employed event-related potentials to examine the time course of neural dynamics involved in emotional word processing. Participants performed a dual-target task in which positive, negative and neutral adjectives were rapidly presented. The early occipital P1 was found larger when elicited by negative words, indicating that the first stage of emotional word processing mainly differentiates between non-threatening and potentially threatening information. The N170 and the early posterior negativity were larger for positive and negative words, reflecting the emotional/non-emotional discrimination stage of word processing. The late positive component not only distinguished emotional words from neutral words, but also differentiated between positive and negative words. This represents the third stage of emotional word processing, the emotion separation. Present results indicated that, similar with the three-stage model of facial expression processing; the neural processing of emotional words can also be divided into three stages. These findings prompt us to believe that the nature of emotion can be analyzed by the brain independent of stimulus type, and that the three-stage scheme may be a common model for emotional information processing in the context of limited attentional resources.

P1 and N170 components distinguish human-like and animal-like makeup stimuli

This study used event-related potentials to investigate the sensitivity of P1 and N170 components to human-like and animal-like makeup stimuli, which were derived from pictures of Peking opera characters. As predicted, human-like makeup stimuli elicited larger P1 and N170 amplitudes than did animal-like makeup stimuli. Interestingly, a right hemisphere advantage was observed for human-like but not for animal-like makeup stimuli. Dipole source analyses of 130-200-ms window showed that the bilateral fusiform face area may contribute to the differential sensitivity of the N170 component in response to human-like and animal-like makeup stimuli. The present study suggests that the amplitudes of both the P1 and the N170 are sensitive for the mouth component of face-like stimuli.