Event-related potentials to unattended changes in facial expressions: detection of regularity violations or encoding of emotions?

Predictability modulates the early neural coding of spatially unattended fearful faces

In this study, we assessed whether predictability affected the early processing of facial expressions. To achieve this, we measured lateralised early- and mid-latency event-related potentials associated with visual processing. Twenty-two participants were shown pairs of bilaterally presented fearful, happy, angry, or scrambled faces. Participants were required to identify angry faces on a spatially attended side whilst ignoring happy, fearful, and scrambled faces. Each block began with the word HAPPY or FEARFUL which informed participants the probability at which these faces would appear. Attention effects were found for the lateralised P1, suggesting that emotions do not modulate the P1 differentially, nor do predictions relating to emotions. Pairwise comparisons demonstrated that, when spatially unattended, unpredicted fearful faces produced larger lateralised N170 amplitudes compared to predicted fearful faces and unpredicted happy faces. Finally, attention towards faces increased lateralised EPN amplitudes, as did both fearful expressions and low predictability. Thus, we demonstrate that the N170 and EPN are sensitive to top-down predictions relating to facial expressions and that low predictability appears to specifically affect the early encoding of fearful faces when unattended, possibly to initiate attentional capture.

Magnetoencephalography reveals impaired sensory gating and change detection in older adults in the somatosensory system

Brain electrophysiological responses can provide information about age-related decline in sensory-cognitive functions with high temporal accuracy. Studies have revealed impairments in early sensory gating and pre-attentive change detection mechanisms in older adults, but no magnetoencephalographic (MEG) studies have been undertaken into both non-attentive and attentive somatosensory functions and their relationship to ageing. Magnetoencephalography was utilized to record cortical somatosensory brain responses in young (20-28 yrs), middle-aged (46-56 yrs), and older adults (64-78 yrs) under active and passive somatosensory oddball conditions. A repeated standard stimulus was occasionally replaced by a deviant stimulus (p = .1), which was an electrical pulse on a different finger. We examined the amplitudes of M50 and M100 responses reflecting sensory gating, and later components reflecting change detection and attention shifting (M190 and M250 for the passive condition, and M200 and M350 for the active condition, respectively). Spatiotemporal cluster-based permutation tests revealed that older adults had significantly larger M100 component amplitudes than young adults for task-irrelevant stimuli in both passive and active condition. Older adults also showed a reduced M250 component and an altered M350 in response to deviant stimuli. The responses of middle-aged adults did not differ from those of younger adults, but this study should be repeated with a larger sample size. By demonstrating changes in both somatosensory gating and attentional shifting mechanisms, our findings extend previous research on the effects of ageing on pre-attentive and attentive brain functions.

Impact of emotional valence on mismatch negativity in the course of cortical face processing

Various aspects of cortical face processing have been studied by assessing event related potentials (ERP). It has been described in the literature that mismatch negativity (MMN), a well-studied ERP, is not only modulated by sensory features but also emotional valence. However, the exact impact of emotion on the temporo-spatial profile of visual MMN during face processing remains inconsistent. By employing a sequential oddball paradigm using both neutral and emotional deviants, we were able to differentiate two distinct vMMN subcomponents. While an early subcomponent at 150-250 ms is elicited by emotional salient facial stimuli, the later subcomponent at 250-400 ms seems to reflect the detection of regularity violations in facial recognition per se, unaffected by emotional salience. Our results suggest that emotional valence is encoded in vMMN signal strength at an early stage of facial processing. Furthermore, we assume that of facial processing consists of temporo-spatially distinct, partially overlapping levels concerning different facial aspects.

Face familiarity revealed by fixational eye movements and fixation-related potentials in free viewing

Event-related potentials (ERPs) and the oculomotor inhibition (OMI) in response to visual transients are known to be sensitive to stimulus properties, attention, and expectation. We have recently found that the OMI is also sensitive to face familiarity. In natural vision, stimulation of the visual cortex is generated primarily by saccades, and it has been recently suggested that fixation-related potentials (FRPs) share similar components with the ERPs. Here, we investigated whether FRPs and microsaccade inhibition (OMI) in free viewing are sensitive to face familiarity. Observers freely watched a slideshow of seven unfamiliar and one familiar facial images presented randomly for 4-s periods, with multiple images per identity. We measured the occipital fixation-related N1 relative to the P1 magnitude as well as the associated fixation-triggered OMI. We found that the average N1-P1 was significantly smaller and the OMI was shorter for the familiar face, compared with any of the seven unfamiliar faces. Moreover, the P1 was suppressed across saccades for the familiar but not for the unfamiliar faces. Our results highlight the sensitivity of the occipital FRPs to stimulus properties such as face familiarity and advance our understanding of the integration process across successive saccades in natural vision.

Emotional expression visual mismatch negativity in children

Detection of changes in facial emotions is crucial to communicate and to rapidly process threats in the environment. This function develops throughout childhood via modulations of the earliest brain responses, such as the P100 and the N170 recorded using electroencephalography. Automatic brain signatures can be measured through expression-related visual mismatch negativity (vMMN), which reflects the processing of unattended changes. While increasing research has investigated vMMN processing in adults, few studies have been conducted on children. Here, a controlled paradigm previously validated was used to disentangle specific responses to emotional deviants (angry face) from that of neutral deviants. Latencies and amplitudes of P100 and N170 both decrease with age, confirming that sensory and face-specific activity is not yet mature in school-aged children. Automatic change detection-related activity is present in children, with a similar vMMN pattern in response to both emotional and neutral deviant stimuli to what previously observed in adults. However, vMMN processing is delayed in children compared to adults and no emotion-specific response is yet observed, suggesting nonmature automatic detection of salient emotional cues. To our knowledge, this is the first study investigating expression-related vMMN in school-aged children, and further investigations are needed to confirm these results.

The Predictive Role of Low Spatial Frequencies in Automatic Face Processing: A Visual Mismatch Negativity Investigation

Visual processing is thought to function in a coarse-to-fine manner. Low spatial frequencies (LSF), conveying coarse information, would be processed early to generate predictions. These LSF-based predictions would facilitate the further integration of high spatial frequencies (HSF), conveying fine details. The predictive role of LSF might be crucial in automatic face processing, where high performance could be explained by an accurate selection of clues in early processing. In the present study, we used a visual Mismatch Negativity (vMMN) paradigm by presenting an unfiltered face as standard stimulus, and the same face filtered in LSF or HSF as deviant, to investigate the predictive role of LSF vs. HSF during automatic face processing. If LSF are critical for predictions, we hypothesize that LSF deviants would elicit less prediction error (i.e., reduced mismatch responses) than HSF deviants. Results show that both LSF and HSF deviants elicited a mismatch response compared with their equivalent in an equiprobable sequence. However, in line with our hypothesis, LSF deviants evoke significantly reduced mismatch responses compared to HSF deviants, particularly at later stages. The difference in mismatch between HSF and LSF conditions involves posterior areas and right fusiform gyrus. Overall, our findings suggest a predictive role of LSF during automatic face processing and a critical involvement of HSF in the fusiform during the conscious detection of changes in faces.

Magnetoencephalography Responses to Unpredictable and Predictable Rare Somatosensory Stimuli in Healthy Adult Humans

Mismatch brain responses to unpredicted rare stimuli are suggested to be a neural indicator of prediction error, but this has rarely been studied in the somatosensory modality. Here, we investigated how the brain responds to unpredictable and predictable rare events. Magnetoencephalography responses were measured in adults frequently presented with somatosensory stimuli (FRE) that were occasionally replaced by two consecutively presented rare stimuli [unpredictable rare stimulus (UR) and predictable rare stimulus (PR); p = 0.1 for each]. The FRE and PR were electrical stimulations administered to either the little finger or the forefinger in a counterbalanced manner between the two conditions. The UR was a simultaneous electrical stimulation to both the forefinger and the little finger (for a smaller subgroup, the UR and FRE were counterbalanced for the stimulus properties). The grand-averaged responses were characterized by two main components: one at 30-100 ms (M55) and the other at 130-230 ms (M150) latency. Source-level analysis was conducted for the primary somatosensory cortex (SI) and the secondary somatosensory cortex (SII). The M55 responses were larger for the UR and PR than for the FRE in both the SI and the SII areas and were larger for the UR than for the PR. For M150, both investigated areas showed increased activity for the UR and the PR compared to the FRE. Interestingly, although the UR was larger in stimulus energy (stimulation of two fingers at the same time) and had a larger prediction error potential than the PR, the M150 responses to these two rare stimuli did not differ in source strength in either the SI or the SII area. The results suggest that M55, but not M150, can possibly be associated with prediction error signals. These findings highlight the need for disentangling prediction error and rareness-related effects in future studies investigating prediction error signals.

Emotional visual mismatch negativity: a joint investigation of social and non-social dimensions in adults with autism

Unusual behaviors and brain activity to socio-emotional stimuli have been reported in Autism Spectrum Disorder (ASD). Atypical reactivity to change and intolerance of uncertainty are also present, but little is known on their possible impact on facial expression processing in autism. The visual mismatch negativity (vMMN) is an electrophysiological response automatically elicited by changing events such as deviant emotional faces presented among regular neutral faces. While vMMN has been found altered in ASD in response to low-level changes in simple stimuli, no study has investigated this response to visual social stimuli. Here two deviant expressions were presented, neutral and angry, embedded in a sequence of repetitive neutral stimuli. vMMN peak analyses were performed for latency and amplitude in early and late time windows. The ASD group presented smaller amplitude of the late vMMN to both neutral and emotional deviants compared to the typically developed adults (TD) group, and only the TD group presented a sustained activity related to emotional change (i.e., angry deviant). Source reconstruction of the vMMNs further revealed that any change processing elicited a reduced activity in ASD group compared to TD in the saliency network, while the specific processing emotional change elicited activity in the temporal region and in the insula. This study confirms atypical change processing in ASD and points to a specific difficulty in the processing of emotional changes, potentially playing a crucial role in social interaction deficits. Nevertheless, these results require to be further replicated with a greater sample size and generalized to other emotional expressions.

Prediction errors in surface segmentation are reflected in the visual mismatch negativity, independently of task and surface features

The visual system quickly registers perceptual regularities in the environment and responds to violations in these patterns. Errors of perceptual prediction are associated with electrocortical modulation, including the visual mismatch negativity (vMMN) and P2 event-related potential. One relatively unexplored question is whether these prediction error signals can encode higher-level properties such as surface segmentation, or whether they are limited to lower-level perceptual features. Using a roving standard paradigm, a triangle surface appeared either behind (featuring amodal contours) or in front of (featuring real contours) a second surface with hole-like windows. A surface layout appeared for two to five repetitions before switching to the other "deviant" layout; lighting and orientation of stimuli varied across presentations while remaining isoluminant. Observers responded when they detected a rare "pinched" triangle, which occasionally appeared. Cortical activity-reflected in mismatch responses affecting the P2-N2 and P300 amplitudes-was sensitive to a change in stimulus layout, when surfaces shifted position in depth, following several repetitions. Specifically, layout deviants led to a more negative P2-N2 complex at posterior electrodes, and greater P300 positivity at central sites. Independently of these signals of a deviant surface layout, further modulations of the P2 encoded differences between layouts and detection of the rare target stimulus. Comparison of the effect of preceding layout repetitions on this prediction error signal suggests that it is all or none and not graded with respect to the number of previous repetitions. We show that within the visual domain, unnoticed and task-irrelevant changes in visual surface segmentation leads to observable electrophysiological signals of prediction error that are dissociable from stimulus-specific encoding and lower-level perceptual processing.

Event-related potentials to task-irrelevant sad faces as a state marker of depression

Negative bias in face processing has been demonstrated in depression, but there are no longitudinal investigations of negative bias in symptom reduction. We recorded event-related potentials (P1 and N170) to task-irrelevant facial expressions in depressed participants who were later provided with a psychological intervention and in never depressed control participants. Follow-up measurements were conducted for the depressed group two and 39 months later. Negative bias was found specifically in the depression group, and was demonstrated as enlarged P1 amplitude to sad faces, which normalized in the follow-up measurements when the participants had fewer symptoms. Because the P1 amplitude recorded at the baseline did not differ between the depression group that recovered and the group that did not recover after the intervention, this brain response did not show potential as a biomarker for treatment response. It could have potential, however, to serve as a state-marker of depression.

Automatic Change Detection of Emotional and Neutral Body Expressions: Evidence From Visual Mismatch Negativity

Rapidly and effectively detecting emotions in others is an important social skill. Since emotions expressed by the face are relatively easy to fake or hide, we often use body language to gauge the genuine emotional state of others. Recent studies suggest that expression-related visual mismatch negativity (vMMN) reflects the automatic processing of emotional changes in facial expression; however, the automatic processing of changes in body expression has not yet been studied systematically. The current study uses an oddball paradigm where neutral body actions served as standard stimuli, while fearful body expressions and other neutral body actions served as two different deviants to define body-related vMMN, and to compare the mechanisms underlying the processing of emotional changes to neutral postural changes. The results show a more negative vMMN amplitude for fear deviants 210-260 ms after stimulus onset which corresponds with the negativity bias that was obtained on the N190 component. In earlier time windows, the vMMN amplitude following the two types of deviant stimuli are identical. Therefore, we present a two-stage model for processing changes in body posture, where changes in body posture are processed in the first 170-210 ms, but emotional changes in the time window of 210-260 ms.

Feature-specific prediction errors for visual mismatch

Predictive coding (PC) theory posits that our brain employs a predictive model of the environment to infer the causes of its sensory inputs. A fundamental but untested prediction of this theory is that the same stimulus should elicit distinct precision weighted prediction errors (pwPEs) when different (feature-specific) predictions are violated, even in the absence of attention. Here, we tested this hypothesis using functional magnetic resonance imaging (fMRI) and a multi-feature roving visual mismatch paradigm where rare changes in either color (red, green), or emotional expression (happy, fearful) of faces elicited pwPE responses in human participants. Using a computational model of learning and inference, we simulated pwPE and prediction trajectories of a Bayes-optimal observer and used these to analyze changes in blood oxygen level dependent (BOLD) responses to changes in color and emotional expression of faces while participants engaged in a distractor task. Controlling for visual attention by eye-tracking, we found pwPE responses to unexpected color changes in the fusiform gyrus. Conversely, unexpected changes of facial emotions elicited pwPE responses in cortico-thalamo-cerebellar structures associated with emotion and theory of mind processing. Predictions pertaining to emotions activated fusiform, occipital and temporal areas. Our results are consistent with a general role of PC across perception, from low-level to complex and socially relevant object features, and suggest that monitoring of the social environment occurs continuously and automatically, even in the absence of attention.

Sex and Physiological Cycles Affect the Automatic Perception of Attractive Opposite-Sex Faces: A Visual Mismatch Negativity Study

Facial attractiveness plays important roles in social interaction. Electrophysiological and neuroimaging studies found several brain areas to be differentially responsive to attractive relative to unattractive faces. However, little is known about the time course of the information processing, especially under the unattended condition. Based on a "cross-modal delayed response" paradigm, the present study aimed to explore the automatic mechanism of facial attractiveness processing of females with different physiological cycles and males, respectively, through recording the event-related potentials in response to (un)attractive opposite-sex faces by two experiments. The attractiveness-related visual mismatch negativity (attractiveness vMMN) in posterior scalp distribution was recorded in both the experiments, which indicated that attractive faces could be processed automatically. And high-attractive opposite-sex faces can elicit larger vMMN in males than females in menstrual period in Study 1, but similar as females in ovulatory period in Study 2. Furthermore, by comparison, the latency of attractiveness vMMN in females with the ovulatory period was the longest. These results indicated as follows: (1) Males were more sensitive to attractive female faces, (2) females in ovulatory period were also attracted by the attractive male faces, (3) the long vMMN latency in females during ovulatory period suggested a special reproductive motivation to avoid being tainted by genes, which takes priority over the breeding motivation.

Automatic Processing of Changes in Facial Emotions in Dysphoria: A Magnetoencephalography Study

It is not known to what extent the automatic encoding and change detection of peripherally presented facial emotion is altered in dysphoria. The negative bias in automatic face processing in particular has rarely been studied. We used magnetoencephalography (MEG) to record automatic brain responses to happy and sad faces in dysphoric (Beck's Depression Inventory ≥ 13) and control participants. Stimuli were presented in a passive oddball condition, which allowed potential negative bias in dysphoria at different stages of face processing (M100, M170, and M300) and alterations of change detection (visual mismatch negativity, vMMN) to be investigated. The magnetic counterpart of the vMMN was elicited at all stages of face processing, indexing automatic deviance detection in facial emotions. The M170 amplitude was modulated by emotion, response amplitudes being larger for sad faces than happy faces. Group differences were found for the M300, and they were indexed by two different interaction effects. At the left occipital region of interest, the dysphoric group had larger amplitudes for sad than happy deviant faces, reflecting negative bias in deviance detection, which was not found in the control group. On the other hand, the dysphoric group showed no vMMN to changes in facial emotions, while the vMMN was observed in the control group at the right occipital region of interest. Our results indicate that there is a negative bias in automatic visual deviance detection, but also a general change detection deficit in dysphoria.

At a Single Glance: Fast Periodic Visual Stimulation Uncovers the Spatio-Temporal Dynamics of Brief Facial Expression Changes in the Human Brain

Detecting brief changes of facial expression is vital for social communication. Yet, how reliably, how fast these changes are detected and how long they are processed in the human brain remain unknown. High-density electroencephalogram (EEG) was recorded in 18 participants presented with a neutral-expression face at a rate of 5.88 Hz (F) for 80 s. Every five faces, the face changed expression to fear, disgust or happiness (different stimulation sequences). The resulting 1.18 Hz (F/5) EEG response and its harmonics objectively indexed detection of a brief change of facial expression. This response was recorded in every participant in a few minutes but was largely reduced for inverted faces, indicating that it reflects high-level processes. Although this response focused on occipito-temporal sites, different expression changes evoked reliably distinct topographical maps, pointing to partly distinct neural generators. These effects were also observed at a faster 12 Hz frequency rate and a lower ratio of expression change (1/9). Time-domain analysis showed that a brief change of expression inserted in a dynamic stimulation sequence elicits specific occipito-temporal responses between 100 and 310 ms, indicating a rapid change detection process followed by a long integration period of facial expression information in the human brain.

Visual Mismatch and Predictive Coding: A Computational Single-Trial ERP Study

Predictive coding (PC) posits that the brain uses a generative model to infer the environmental causes of its sensory data and uses precision-weighted prediction errors (pwPEs) to continuously update this model. While supported by much circumstantial evidence, experimental tests grounded in formal trial-by-trial predictions are rare. One partial exception is event-related potential (ERP) studies of the auditory mismatch negativity (MMN), where computational models have found signatures of pwPEs and related model-updating processes. Here, we tested this hypothesis in the visual domain, examining possible links between visual mismatch responses and pwPEs. We used a novel visual "roving standard" paradigm to elicit mismatch responses in humans (of both sexes) by unexpected changes in either color or emotional expression of faces. Using a hierarchical Bayesian model, we simulated pwPE trajectories of a Bayes-optimal observer and used these to conduct a comprehensive trial-by-trial analysis across the time × sensor space. We found significant modulation of brain activity by both color and emotion pwPEs. The scalp distribution and timing of these single-trial pwPE responses were in agreement with visual mismatch responses obtained by traditional averaging and subtraction (deviant-minus-standard) approaches. Finally, we compared the Bayesian model to a more classical change model of MMN. Model comparison revealed that trial-wise pwPEs explained the observed mismatch responses better than categorical change detection. Our results suggest that visual mismatch responses reflect trial-wise pwPEs, as postulated by PC. These findings go beyond classical ERP analyses of visual mismatch and illustrate the utility of computational analyses for studying automatic perceptual processes.SIGNIFICANCE STATEMENT Human perception is thought to rely on a predictive model of the environment that is updated via precision-weighted prediction errors (pwPEs) when events violate expectations. This "predictive coding" view is supported by studies of the auditory mismatch negativity brain potential. However, it is less well known whether visual perception of mismatch relies on similar processes. Here we combined computational modeling and electroencephalography to test whether visual mismatch responses reflected trial-by-trial pwPEs. Applying a Bayesian model to series of face stimuli that violated expectations about color or emotional expression, we found significant modulation of brain activity by both color and emotion pwPEs. A categorical change detection model performed less convincingly. Our findings support the predictive coding interpretation of visual mismatch responses.

Facial Expression Related vMMN: Disentangling Emotional from Neutral Change Detection

Detection of changes in facial emotional expressions is crucial to communicate and to rapidly and automatically process possible threats in the environment. Recent studies suggest that expression-related visual mismatch negativity (vMMN) reflects automatic processing of emotional changes. In the present study we used a controlled paradigm to investigate the specificity of emotional change-detection. In order to disentangle specific responses to emotional deviants from that of neutral deviants, we presented neutral expression as standard stimulus (p = 0.80) and both angry and neutral expressions as deviants (p = 0.10, each). In addition to an oddball sequence, an equiprobable sequence was presented, to control for refractoriness and low-level differences. Our results showed that in an early time window (100–200 ms), the controlled vMMN was greater than the oddball vMMN only for the angry deviant, suggesting the importance of controlling for refractoriness and stimulus physical features in emotion related studies. Within the controlled vMMN, angry and neutral deviants both elicited early and late peaks occurring at 140 and 310 ms, respectively, but only the emotional vMMN presented sustained amplitude after each peak. By directly comparing responses to emotional and neutral deviants, our study provides evidence of specific activity reflecting the automatic detection of emotional change. This differs from broader “visual” change processing, and suggests the involvement of two partially-distinct pre-attentional systems in the detection of changes in facial expressions.

Influence of attention on bimodal integration during emotional change decoding: ERP evidence

Recent findings on audiovisual emotional interactions suggest that selective attention affects cross-sensory interaction from an early processing stage. However, the influence of attention manipulation on facial-vocal integration during emotional change perception is still elusive at this point. To address this issue, we asked participants to detect emotional changes conveyed by prosodies (vocal task) or facial expressions (facial task) while facial, vocal, and facial-vocal expressions were presented. At the same time, behavioral responses and electroencephalogram (EEG) were recorded. Behavioral results showed that bimodal emotional changes were detected with shorter response latencies compared to each unimodal condition, suggesting that bimodal emotional cues facilitated emotional change detection. Moreover, while the P3 amplitudes were larger for the bimodal change condition than for the sum of the two unimodal conditions regardless of attention direction, the N1 amplitudes were larger for the bimodal emotional change condition than for the sum of the two unimodal conditions under the attend-voice condition, but not under the attend-face condition. These findings suggest that selective attention modulates facial-vocal integration during emotional change perception in early sensory processing, but not in late cognitive processing stages.

Automatic Change Detection to Facial Expressions in Adolescents: Evidence from Visual Mismatch Negativity Responses

Adolescence is a critical period for the neurodevelopment of social-emotional processing, wherein the automatic detection of changes in facial expressions is crucial for the development of interpersonal communication. Two groups of participants (an adolescent group and an adult group) were recruited to complete an emotional oddball task featuring on happy and one fearful condition. The measurement of event-related potential was carried out via electroencephalography and electrooculography recording, to detect visual mismatch negativity (vMMN) with regard to the automatic detection of changes in facial expressions between the two age groups. The current findings demonstrated that the adolescent group featured more negative vMMN amplitudes than the adult group in the fronto-central region during the 120–200 ms interval. During the time window of 370–450 ms, only the adult group showed better automatic processing on fearful faces than happy faces. The present study indicated that adolescent’s posses stronger automatic detection of changes in emotional expression relative to adults, and sheds light on the neurodevelopment of automatic processes concerning social-emotional information.

Neurophysiological modulation of rapid emotional face processing is associated with impulsivity traits

Background

Sensori-perceptual processing of emotional stimuli under attentive conditions effectively prevents response disinhibition. This is observed saliently in low-impulsive people, because of their high sensitivity to warning signals, such as emotional faces. Results from human neurophysiological studies have been used to develop a dual detector model for early sensori-perceptual processing. A transient detector mechanism is related to automatic neurophysiological arousal in response to warning signals, which is reflected by early frontal event-related potential effects. The memory-based detector mechanism is associated with subsequent mismatch negativity (MMN), which reflects a short-term memory trace of signals. Based on previous findings, we predicted that impulsivity affects functional associations among the dual detector mechanisms, and modulates early frontal and/or MMN activities. In the present study, we recorded electroencephalograms for twenty-one healthy adults using a visual oddball paradigm with neutral faces as frequent stimuli, and angry and happy faces as infrequent stimuli. We measured the impulsivity traits by a self-report scale (the Barratt Impulsiveness Scale, 11th version).

Results

Main findings were that only happy faces increased early frontal negativity and subsequent occipital visual MMN (vMMN) for emotional change, and these neurophysiological effects positively correlated with each other in a temporally causal manner. However, an impulsivity sub-trait positively correlated selectively with vMMN for the happy faces.

Conclusion

These findings demonstrate that higher impulsivity is associated with attenuated vMMN for emotional change detection in healthy populations, potentially because of weakened fronto-occipital functional connection that is responsible for the dual detector mechanism.

Electronic supplementary material

The online version of this article (doi:10.1186/s12868-015-0223-x) contains supplementary material, which is available to authorized users.

Fluid Intelligence and Automatic Neural Processes in Facial Expression Perception: An Event-Related Potential Study

The relationship between human fluid intelligence and social-emotional abilities has been a topic of considerable interest. The current study investigated whether adolescents with different intellectual levels had different automatic neural processing of facial expressions. Two groups of adolescent males were enrolled: a high IQ group and an average IQ group. Age and parental socioeconomic status were matched between the two groups. Participants counted the numbers of the central cross changes while paired facial expressions were presented bilaterally in an oddball paradigm. There were two experimental conditions: a happy condition, in which neutral expressions were standard stimuli (p = 0.8) and happy expressions were deviant stimuli (p = 0.2), and a fearful condition, in which neutral expressions were standard stimuli (p = 0.8) and fearful expressions were deviant stimuli (p = 0.2). Participants were required to concentrate on the primary task of counting the central cross changes and to ignore the expressions to ensure that facial expression processing was automatic. Event-related potentials (ERPs) were obtained during the tasks. The visual mismatch negativity (vMMN) components were analyzed to index the automatic neural processing of facial expressions. For the early vMMN (50-130 ms), the high IQ group showed more negative vMMN amplitudes than the average IQ group in the happy condition. For the late vMMN (320-450 ms), the high IQ group had greater vMMN responses than the average IQ group over frontal and occipito-temporal areas in the fearful condition, and the average IQ group evoked larger vMMN amplitudes than the high IQ group over occipito-temporal areas in the happy condition. The present study elucidated the close relationships between fluid intelligence and pre-attentive change detection on social-emotional information.

Event-related potentials elicited by social commerce and electronic-commerce reviews

There is an increasing interest regarding the use of electroencephalography (EEG) in social commerce and electronic commerce (e-commerce) research. There are several reviews in the field of social commerce or e-commerce; these have great potential value and mining them is fundamental and significant. To our knowledge, EEG is rarely applied to study these. In this study, we examined the neural correlates of social commerce reviews (SCRs) and e-commerce reviews (ECRs) by using them as stimuli to evoke event-related potentials. All SCRs were from friends through a social media platform, whereas ECRs were from strangers through an e-commerce platform. The experimental design was similar to that of a priming paradigm, and included 40 pairs of stimuli consisting of product information (prime stimulus) and reviews (target stimulus). The results showed that the P300 component was successfully evoked by SCR and ECR stimuli. Moreover, the P300 components elicited by SCRs had higher amplitudes than those elicited by ECRs. These findings indicate that participants paid more attention to SCRs than to ECRs. In addition, the associations between neural responses and reviews in social commerce have the potential to assist companies in studying consumer behaviors, thus permitting them to enhance their social commerce strategies.

Elementary sensory deficits in schizophrenia indexed by impaired visual mismatch negativity

INTRODUCTION

Mismatch negativity (MMN) is an automatic brain response to unexpected events. It represents a prediction error (PE) response, reflecting the difference between the sensory input and predictions. While deficits in auditory MMN are well known in schizophrenia, only few studies investigated impairments in predictive visual processing in schizophrenia. These studies used complex stimuli such as motion direction and emotional facial expressions. Here we studied whether automatic predictive processing of elementary features such as orientation is also impaired in schizophrenia.

METHODS

Altogether 28 patients with schizophrenia and 27 healthy controls matched in age, gender, and education participated in the study. EEG was recorded using 128 channels in the two experimental blocks. Using an oddball paradigm, horizontal stripes of Gabor patches were presented as frequent standards and vertical stripes as rare deviants in one block. Stimulus probabilities were swapped in the other block. Mismatch responses were obtained by subtracting responses to standard from those to deviant stimuli.

RESULTS

We found significant mismatch responses in healthy controls but not in patients in the prefrontal and occipital-parietal regions in the 90-200ms interval. Furthermore patients showed significantly decreased deviant minus standard difference waveforms relative to controls in the same regions with moderate to large effect sizes.

CONCLUSIONS

Our findings demonstrate that predictive processing of unattended low-level visual features such as orientation is impaired in schizophrenia. Our results complement reports of sensory deficits found in tasks requiring attentive processing and suggest that deficits are present in automatic visual sensory processes putatively mediated by glutamatergic functioning.

The integration of facial and vocal cues during emotional change perception: EEG markers

The ability to detect emotional changes is of primary importance for social living. Though emotional signals are often conveyed by multiple modalities, how emotional changes in vocal and facial modalities integrate into a unified percept has yet to be directly investigated. To address this issue, we asked participants to detect emotional changes delivered by facial, vocal and facial-vocal expressions while behavioral responses and electroencephalogram were recorded. Behavioral results showed that bimodal emotional changes were detected with higher accuracy and shorter response latencies compared with each unimodal condition. Moreover, the detection of emotional change, regardless of modalities, was associated with enhanced amplitudes in the N2 and P3 component, as well as greater theta synchronization. More importantly, the P3 amplitudes and theta synchronization were larger for the bimodal emotional change condition than for the sum of the two unimodal conditions. The superadditive responses in P3 amplitudes and theta synchronization were both positively correlated with the magnitude of the bimodal superadditivity in accuracy. These behavioral and electrophysiological data consistently illustrated an effect of audiovisual integration during the detection of emotional changes, which is most likely mediated by the P3 activity and theta oscillations in brain responses.

Emotional context facilitates cortical prediction error responses

In the predictive coding framework, mismatch negativity (MMN) is regarded a correlate of the prediction error that occurs when top-down predictions conflict with bottom-up sensory inputs. Expression-related MMN is a relatively novel construct thought to reflect a prediction error specific to emotional processing, which, however, has not yet been tested directly. Our paradigm includes both neutral and emotional deviants, thereby allowing for investigating whether expression-related MMN is emotion-specific or unspecifically arises from violations of a given sequence. Twenty healthy participants completed a visual sequence oddball task where they were presented with (1) sequence deviants, (2) emotional sequence deviants, and (3) emotional deviants. Mismatch components were assessed at ventral occipitotemporal scalp sites and analyzed regarding their amplitudes, spatiotemporal profiles, and neuronal sources. Expression-related MMN could be clearly separated from its neutral counterpart in all investigated aspects. Specifically, expression-related MMN showed enhanced amplitude, shorter latency, and different neuronal sources. Our results, therefore, provide converging evidence for a quantitative specificity of expression-related MMN and seems to provide an opportunity to study prediction error during preattentive emotional processing. Our neurophysiological evidence ultimately suggests that a basic cognitive operator, the prediction error, is enhanced at the cortical level by processing of emotionally salient stimuli.

Is empathy necessary to comprehend the emotional faces? The empathic effect on attentional mechanisms (eye movements), cortical correlates (N200 event-related potentials) and facial behaviour (electromyography) in face processing

The present research explored the effect of social empathy on processing emotional facial expressions. Previous evidence suggested a close relationship between emotional empathy and both the ability to detect facial emotions and the attentional mechanisms involved. A multi-measure approach was adopted: we investigated the association between trait empathy (Balanced Emotional Empathy Scale) and individuals' performance (response times; RTs), attentional mechanisms (eye movements; number and duration of fixations), correlates of cortical activation (event-related potential (ERP) N200 component), and facial responsiveness (facial zygomatic and corrugator activity). Trait empathy was found to affect face detection performance (reduced RTs), attentional processes (more scanning eye movements in specific areas of interest), ERP salience effect (increased N200 amplitude), and electromyographic activity (more facial responses). A second important result was the demonstration of strong, direct correlations among these measures. We suggest that empathy may function as a social facilitator of the processes underlying the detection of facial emotion, and a general "facial response effect" is proposed to explain these results. We assumed that empathy influences cognitive and the facial responsiveness, such that empathic individuals are more skilful in processing facial emotion.

Visual mismatch negativity: a predictive coding view

An increasing number of studies investigate the visual mismatch negativity (vMMN) or use the vMMN as a tool to probe various aspects of human cognition. This paper reviews the theoretical underpinnings of vMMN in the light of methodological considerations and provides recommendations for measuring and interpreting the vMMN. The following key issues are discussed from the experimentalist's point of view in a predictive coding framework: (1) experimental protocols and procedures to control "refractoriness" effects; (2) methods to control attention; (3) vMMN and veridical perception.

Low-rank approximation based non-negative multi-way array decomposition on event-related potentials

Non-negative tensor factorization (NTF) has been successfully applied to analyze event-related potentials (ERPs), and shown superiority in terms of capturing multi-domain features. However, the time-frequency representation of ERPs by higher-order tensors are usually large-scale, which prevents the popularity of most tensor factorization algorithms. To overcome this issue, we introduce a non-negative canonical polyadic decomposition (NCPD) based on low-rank approximation (LRA) and hierarchical alternating least square (HALS) techniques. We applied NCPD (LRAHALS and benchmark HALS) and CPD to extract multi-domain features of a visual ERP. The features and components extracted by LRAHALS NCPD and HALS NCPD were very similar, but LRAHALS NCPD was 70 times faster than HALS NCPD. Moreover, the desired multi-domain feature of the ERP by NCPD showed a significant group difference (control versus depressed participants) and a difference in emotion processing (fearful versus happy faces). This was more satisfactory than that by CPD, which revealed only a group difference.