Consciousness modulates the automatic change detection of masked emotional faces: Evidence from visual mismatch negativity

Prioritization of social information processing: Eye gaze elicits earlier vMMN than arrows

Numerous research studies have demonstrated that eye gaze and arrows act as cues that automatically guide spatial attention. However, it remains uncertain whether the attention shifts triggered by these two types of stimuli vary in terms of automatic processing mechanisms. In our current investigation, we employed an equal probability paradigm to explore the likenesses and distinctions in the neural mechanisms of automatic processing for eye gaze and arrows in non-attentive conditions, using visual mismatch negative (vMMN) as an indicator of automatic processing. The sample size comprised 17 participants. The results indicated a significant interaction between time duration, stimulus material, and stimulus type. The findings demonstrated that both eye gaze and arrows were processed automatically, triggering an early vMMN, although with temporal variations. The vMMN for eye gaze occurred between 180 and 220 ms, whereas for arrows it ranged from 235 to 275 ms. Moreover, arrow stimuli produced a more pronounced vMMN amplitude. The earlier vMMN response to eye gaze compared with arrows implies the specificity and precedence of social information processing associated with eye gaze over the processing of nonsocial information with arrows. However, arrow could potentially elicit a stronger vMMN because of their heightened salience compared to the background, and the expansion of attention focusing might amplify the vMMN impact. This study offers insights into the similarities and differences in attention processing of social and non-social information under unattended conditions from the perspective of automatic processing.

Automatic processing of unattended mean emotion: Evidence from visual mismatch responses

The mean emotion from multiple facial expressions can be extracted rapidly and precisely. However, it remains debated whether mean emotion processing is automatic which can occur under no attention. To address this question, we used a passive oddball paradigm and recorded event-related brain potentials when participants discriminated the changes in the central fixation while a set of four faces was presented in the periphery. The face set consisted of one happy and three angry expressions (mean negative) or one angry and three happy expressions (mean positive), and the mean negative and mean positive face sets were shown with a probability of 20% (deviant) and 80% (standard) respectively in the sequence, or the vice versa. The cluster-based permutation analyses showed that the visual mismatch negativity started early at around 92 ms and was also observed in later time windows when the mean emotion was negative, while a mismatch positivity was observed at around 168- 266 ms when the mean emotion was positive. The results suggest that there might be different mechanisms underlying the processing of mean negative and mean positive emotions. More importantly, the brain can detect the changes in the mean emotion automatically, and ensemble coding for multiple facial expressions can occur in an automatic fashion without attention.

Facial feedback manipulation influences the automatic detection of unexpected emotional body expressions

Unexpected or changing facial expressions are known to be able to engage more automatic processing than frequently occurring facial expressions, thereby inducing a neural differential wave response known as expression mismatch negativity (EMMN). Recent studies have shown that EMMN can be modulated by the observer's facial feedback (i.e., feedback from their own facial movements). A similar EMMN activity has been discovered for body expressions, but thus far only a few emotion types have been investigated. It is unknown whether the EMMNs evoked by body expressions can be influenced by facial feedback. To explore this question, we recorded EEG activity of 29 participants in the reverse oddball paradigm. Here two unexamined categories of body expressions were presented, happy and sad, placed in two paired stimulus sequences: in one the happy body was presented with a probability of 80% (standards) while the sad body was presented with a probability of 20% (deviants), and in the other the probabilities were reversed. The facial feedback was manipulated by different pen holding conditions (i.e., participants holding the pen with the teeth, lips, or nondominant hand). The nonparametric cluster permutation test revealed significant happy and sad body-related EMMN (bEMMN) activities. The happy-bEMMN were more negative than sad-bEMMN within the range of 100-150 ms. Additionally, the bEMMN amplitude of both emotions is modulated by the facial feedback conditions. These results expand the range of emotional types applicable to bEMMN and provide evidence for the validity of the facial feedback hypothesis across emotional carriers.

The role of attention control in visual mismatch negativity (vMMN) studies

The detection of unattended visual changes is investigated by the visual mismatch negativity (vMMN) component of event-related potentials (ERPs). The vMMN is measured as the difference between the ERPs to infrequent (deviant) and frequent (standard) stimuli irrelevant to the ongoing task. In the present study, we used human faces expressing different emotions as deviants and standards. In such studies, participants perform various tasks, so their attention is diverted from the vMMN-related stimuli. If such tasks vary in their attentional demand, they might influence the outcome of vMMN studies. In this study, we compared four kinds of frequently used tasks: (1) a tracking task that demanded continuous performance, (2) a detection task where the target stimuli appeared at any time, (3) a detection task where target stimuli appeared only in the inter-stimulus intervals, and (4) a task where target stimuli were members of the stimulus sequence. This fourth task elicited robust vMMN, while in the other three tasks, deviant stimuli elicited moderate posterior negativity (vMMN). We concluded that the ongoing task had a marked influence on vMMN; thus, it is important to consider this effect in vMMN studies.

Links between social avoidance and frontal alpha asymmetry during processing emotional facial stimuli: An exploratory study

Individuals who are socially avoidant actively remove themselves from opportunities for social interaction and have a strong desire for solitude. Although social avoidance is associated with a host of adjustment difficulties, its neural substrates remain under-explored. To address this gap, we conducted an exploratory study to compare electroencephalography (EEG) frontal alpha asymmetry (FAA) scores during processing emotional facial stimuli in socially avoidant and non-withdrawn comparison individuals. From an original sample of N = 384 undergraduate students, 25 avoidant and 27 comparison individuals were identified. For this subset of participants, EEG modulations and self-reported experience ratings during a picture processing task were assessed. Among the results, the socially avoidant group's ratings of positive stimuli were significantly lower than the non-withdrawn comparison group. The socially avoidant group also had significantly lower FAA scores in response to positive stimuli than the comparison group. Further, asymmetry scores of the comparison group in the positive conditions were higher than in the negative and neutral conditions. However, there were no significant differences between these three conditions in the socially avoidant group. Our results suggest that socially avoidant individuals may eschew interpersonal relationships because of a relatively greater right hemisphere cortical activity, which may contribute to a withdrawal motivation when confronted with negative emotional stimuli in social contexts.

Automatic change detection: Mismatch negativity and the now-classic Rensink, O’Reagan, and Clark (1997) stimuli

Change blindness experiments had demonstrated that detection of significant changes in natural images is extremely difficult when brief blank fields are placed between alternating displays of an original and a modified scene. On the other hand, research on the visual mismatch negativity (vMMN) component of the event-related potentials (ERPs) identified sensitivity to events (deviants) different from the regularity of stimulus sequences (standards), even if the deviant and standard events are non-attended. The present study sought to investigate the apparent controversy between the experience under the change blindness paradigm and the ERP results. To this end, the stimulus of Rensink, O’Reagen, and Clark (1997) was adapted to a passive oddball ERP paradigm to investigate the underlying processing differences between the standard (original) and deviant (altered) stimuli measured in 22 subjects. Posterior negativity within the 280–330 ms latency range emerged as the difference between ERPs elicited by standard and deviant stimuli, identified as visual mismatch negativity (vMMN). These results raise the possibility that change blindness is not based on the lack of detailed visual representations or the deficiency of comparing two representations. However, effective discrimination of the two scene versions requires considerable frequency differences between them.

Configural but Not Featural Face Information Is Associated With Automatic Processing

Configural face processing precedes featural face processing under the face-attended condition, but their temporal sequence in the absence of attention is unclear. The present study investigated this issue by recording visual mismatch negativity (vMMN), which indicates the automatic processing of visual information under unattended conditions. Participants performed a central cross size change detection task, in which random sequences of faces were presented peripherally, in an oddball paradigm. In Experiment 1, configural and featural faces (deviant stimuli) were presented infrequently among original faces (standard stimuli). In Experiment 2, configural faces were presented infrequently among featural faces, or vice versa. The occipital-temporal vMMN emerged in the 200–360 ms latency range for configural, but not featural, face information. More specifically, configural face information elicited a substantial vMMN component in the 200–360 ms range in Experiment 1. This result was replicated in the 320–360 ms range in Experiment 2, especially in the right hemisphere. These results suggest that configural, but not featural, face information is associated with automatic processing and provides new electrophysiological evidence for the different mechanisms underlying configural and featural face processing under unattended conditions.

The Predictive Role of Low Spatial Frequencies in Automatic Face Processing: A Visual Mismatch Negativity Investigation

Visual processing is thought to function in a coarse-to-fine manner. Low spatial frequencies (LSF), conveying coarse information, would be processed early to generate predictions. These LSF-based predictions would facilitate the further integration of high spatial frequencies (HSF), conveying fine details. The predictive role of LSF might be crucial in automatic face processing, where high performance could be explained by an accurate selection of clues in early processing. In the present study, we used a visual Mismatch Negativity (vMMN) paradigm by presenting an unfiltered face as standard stimulus, and the same face filtered in LSF or HSF as deviant, to investigate the predictive role of LSF vs. HSF during automatic face processing. If LSF are critical for predictions, we hypothesize that LSF deviants would elicit less prediction error (i.e., reduced mismatch responses) than HSF deviants. Results show that both LSF and HSF deviants elicited a mismatch response compared with their equivalent in an equiprobable sequence. However, in line with our hypothesis, LSF deviants evoke significantly reduced mismatch responses compared to HSF deviants, particularly at later stages. The difference in mismatch between HSF and LSF conditions involves posterior areas and right fusiform gyrus. Overall, our findings suggest a predictive role of LSF during automatic face processing and a critical involvement of HSF in the fusiform during the conscious detection of changes in faces.

Visual Mismatch Negativity Reflects Enhanced Response to the Deviant: Evidence From Event-Related Potentials and Electroencephalogram Time-Frequency Analysis

Automatic detection of information changes in the visual environment is crucial for individual survival. Researchers use the oddball paradigm to study the brain’s response to frequently presented (standard) stimuli and occasionally presented (deviant) stimuli. The component that can be observed in the difference wave is called visual mismatch negativity (vMMN), which is obtained by subtracting event-related potentials (ERPs) evoked by the deviant from ERPs evoked by the standard. There are three hypotheses to explain the vMMN. The sensory fatigue (or refractoriness) hypothesis considers that weakened neural activity caused by repetition results in decreased ERPs of the standard. The memory trace hypothesis proposes that vMMN results from increased responses to the deviant. The predictive coding hypothesis attributes the difference to enhanced responses for deviants and suppression for standards. However, when distinguishing between these effects, previous researchers did not consider the effect of low-level features on the vMMN. In this experiment, we used face sequences composed of different emotions (e.g., neutral and fearful face) and presented an oddball sequence, a reverse oddball sequence, and an equiprobable sequence to participants. The deviant of the oddball sequence was subtracted from the standard of the oddball sequence, the reverse oddball sequence, and the same type of stimulus of the equiprobable sequence to get oddball-vMMN (vMMN1), reverse oddball-vMMN (vMMN2), and equiprobable-vMMN (vMMN3), respectively. The results showed no significant difference between vMMN2 and vMMN3 in 100–350 ms following stimulus onset, while the vMMN effect was significant, indicating that the probability of the standard did not affect vMMN, which supported the memory trace hypothesis. Additionally, the fearful-related vMMN were more negative than the neutral-related vMMN within the range of 100–150 ms, suggesting a negative bias. We analyzed the source location of different vMMNs. There was no significant difference in brain regions between different vMMNs. Time-frequency analysis showed that the deviant had stronger theta-band oscillatory than the standard (visual mismatch oscillatory responses, vMORs). However, there was no difference between vMORs2 and vMORs3, indicating that vMORs reflect an enhanced response to the deviant in terms of neural oscillation, supporting the memory trace hypothesis.

The automatic detection of unexpected emotion and neutral body postures: A visual mismatch negativity study

The ability to automatically detect emotional changes in the environment is crucial for social interaction. In the visual system, expression-related mismatch negativity (EMMN) reflects the automatic processing of emotional changes in facial expression. However, body postures also carry visual emotional information that can be recognized effectively and processed automatically, although their processing mechanism remains unknown. In this study, the reverse oddball paradigm was used to investigate the mismatch responses of unexpected fear and neutral body postures. The nonparametric cluster permutation test revealed significant fear and neutral visual mismatch negativity (vMMN) activities, and the fear-related vMMN was enhanced prior (130-230 ms) to the neutral vMMN (180-230 ms). The body-sensitive N190 component may partially account for the vMMN obtained in this study. The fearful body posture evoked a greater N190 response over the neutral body, and amplitudes of N190 were more negative in the deviant condition than the standard condition. Additionally, the body-related visual mismatch oscillatory responses were associated with enhancement of the alpha band oscillation, especially for the fearful body posture. These results expanded the applicable scope of body posture cues corresponding to mismatch signals, objectively defined the electrophysiological activities evoked, and revealed the processing bias toward negative emotion.

The Effect of the Graphic Structures of Humanoid Robot on N200 and P300 Potentials

Humanoid robots are widely used in brain computer interface (BCI). Using a humanoid robot stimulus could increase the amplitude of event-related potentials (ERPs), which improves BCI performance. Since a humanoid robot contains many human elements, the element that increases the ERPs amplitude is unclear, and how to test the effect of it on the brain is a problem. This study used different graphic structures of an NAO humanoid robot to design three types of robot stimuli: a global robot, its local information, and its topological action. Ten subjects first conducted an odd-ball-based BCI (OD-BCI) by applying these stimuli. Then, they accomplished a delayed matching-to-sample task (DMST) that was used to specialize the encoding and retrieval phases of the OD-BCI task. In the retrieval phase of the DMST, the global stimulus induces the largest N200 and P300 potentials with the shortest latencies in the frontal, central, and occipital areas. This finding is in accordance with the P300 and classification performance of the OD-BCI task. When induced by the local stimulus, the subjects responded faster and more accurately in the retrieval phase of the DMST than in the other two conditions, indicating that the local stimulus improved the subject's responses. These results indicate that the OD-BCI task causes subject's retrieval work when the subject recognizes and outputs the stimulus. The global stimulus that contains topological and local elements could make brain react faster and induce larger ERPs, this finding could be used during the development of visual stimuli to improve BCI performance.