Visual mismatch negativity and its importance in visual cognitive sciences

Rare incompatible stimuli evoke visual mismatch negativity in a flanker task

In the flanker task, the behavioral performance for incompatible stimuli is worse in the mostly compatible (rare) condition than in the equiprobable condition. Furthermore, incompatible stimuli evoke visual mismatch negativity (VMMN) when comparing the rare and equiprobable conditions. Compatible and incompatible stimuli differ in terms of their shape and type. This study aimed to examine whether VMMN evoked by rare incompatible stimuli were associated with the shape or type of the stimulus. In a modified version of the flanker task, stimuli were manipulated by two shapes (typical or peculiar) and two types (compatible or incompatible): typical compatible stimuli (< <  <  <  < and >  >  >  > >), typical incompatible stimuli (> >  <  >  > and <  <  >  < <), peculiar compatible stimuli (+ <  <  <  + and +  >  >  > +), and peculiar incompatible stimuli (+ >  <  >  + and +  <  >  < +). In the rare condition, typical incompatible, peculiar compatible, and peculiar incompatible stimuli were presented with a probability of 10%, whereas all the stimuli were presented equally in the equiprobable condition. Right posterior negativity from 200 to 250 ms was significantly more negative in the rare condition than in the equiprobable condition for typical and peculiar incompatible stimuli; however, this difference was not observed for peculiar compatible stimuli. VMMN was significantly more negative for typical and peculiar incompatible stimuli than for peculiar compatible stimuli, and was not significantly different between typical and peculiar incompatible stimuli. These findings suggest that VMMN for incompatible stimuli is associated with the type rather than the shape of the stimulus.

The effects of positions on deviant processing in mostly incompatible blocks in the flanker task

It is assumed that focused attention is induced by mostly incompatible (MI) blocks in the flanker task. This study aimed to examine the differences in deviant processing between positions of a stimulus in MI blocks. Thirty-nine adults participated in this study. Compatible and incompatible stimuli were classified into three types: typical (central and surrounding colors: black), central-deviant (central: red; surrounding: black), and surrounding-deviant (central: black; surrounding: red). Rare and equiprobable conditions were set for MI blocks. Central- and surrounding-deviant stimuli were presented with low probabilities in the rare condition and with identical probabilities to that of typical stimuli in the equiprobable condition. Deviant processing was evaluated by comparing between event-related potentials in rare and equiprobable conditions. The posterior negativity from 120 to 170 ms (i.e., N1) for central-deviant stimuli was significantly more negative in the rare condition than in the equiprobable condition, whereas there was no difference for surrounding-deviant stimuli. Conversely, the posterior negativity from 180 to 230 ms for both stimuli was significantly more negative in the rare condition than in the equiprobable condition, and the difference (i.e., visual mismatch negativity) was similar in central- and surrounding-deviant stimuli. These findings suggest that focused attention induced by MI blocks leads to differences in deviant processing between central and surrounding areas during the N1 time range. Therefore, evaluations of deviant processing can help examine processing in central and surrounding areas independently and are valuable for understanding cognitive control mechanisms in the flanker tasks.

Enhanced electrophysiological responses to explicitly predicted and pre-imagined inputs, with confirmation from online decoding with neuro-feedback

Neural responses to sensory inputs can scale with the likelihood of encountering the input. This is consistent with the predictive coding framework, in that the human brain is expected to be less responsive to predicted inputs. Typically, however, prediction is not explicitly measured. It is inferred from the probability of encountering an event. When an input is explicitly predicted, responses to predicted inputs can be enhanced. Here, we ask if this effect can be ascribed to a generic priming effect, from pre-cogitating about one of two possible inputs. Consistent with this, we find that P300s (a relatively late event-related potential measured with electroencephalography) are greater for explicitly predicted audio and visual inputs, and that this effect cannot be distinguished from a priming effect from pre-imagining audio or visual presentations. Evidence indicates that participants engaged in pre-imagining presentations, as we were able to decode online what type of presentation (audio or visual) they were imagining with a high success rate (approx. 73%), and we encouraged compliance with neuro-feedback regarding this success rate. Our data confirm that human cortex can be more responsive to inputs that have been subject to pre-cogitation-including explicit predictions. This highlights that while anticipatory processes can reduce responding to likely inputs, they can also enhance responding to explicitly predicted inputs.

A multimodal cortical network of sensory expectation violation revealed by fMRI

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

Brain activity characteristics of RGB stimulus: an EEG study

The perception of color is a fundamental cognitive feature of our psychological experience, with an essential role in many aspects of human behavior. Several studies used magnetoencephalography, functional magnetic resonance imaging, and electroencephalography (EEG) approaches to investigate color perception. Their methods includes the event-related potential and spectral power activity of different color spaces, such as Derrington-Krauskopf-Lennie and red-green-blue (RGB), in addition to exploring the psychological and emotional effects of colors. However, we found insufficient studies in RGB space that considered combining all aspects of EEG signals. Thus, in the present study, focusing on RGB stimuli and using a data-driven approach, we investigated significant differences in the perception of colors. Our findings show that beta oscillation of green compared to red and blue colors occurs in early sensory periods with a latency shifting in the occipital region. Furthermore, in the occipital region, the theta power of the blue color decreases noticeably compared to the other colors. Concurrently, in the prefrontal area, we observed an increase in phase consistency in response to the green color, while the blue color showed a decrease. Therefore, our results can be used to interpret the brain activity mechanism of color perception in RGB color space and to choose suitable colors for more efficient performance in cognitive activities.

The P3 indexes the distance between perceived and target image

Visual recognition requires inferring the similarity between a perceived object and a mental target. However, a measure of similarity is difficult to determine when it comes to complex stimuli such as faces. Indeed, people may notice someone "looks like" a familiar face, but find it hard to describe on the basis of what features such a comparison is based. Previous work shows that the number of similar visual elements between a face pictogram and a memorized target correlates with the P300 amplitude in the visual evoked potential. Here, we redefine similarity as the distance inferred from a latent space learned using a state-of-the-art generative adversarial neural network (GAN). A rapid serial visual presentation experiment was conducted with oddball images generated at varying distances from the target to determine how P300 amplitude related to GAN-derived distances. The results showed that distance-to-target was monotonically related to the P300, showing perceptual identification was associated with smooth, drifting image similarity. Furthermore, regression modeling indicated that while the P3a and P3b sub-components had distinct responses in location, time, and amplitude, they were similarly related to target distance. The work demonstrates that the P300 indexes the distance between perceived and target image in smooth, natural, and complex visual stimuli and shows that GANs present a novel modeling methodology for studying the relationships between stimuli, perception, and recognition.

The role of attention control in visual mismatch negativity (vMMN) studies

The detection of unattended visual changes is investigated by the visual mismatch negativity (vMMN) component of event-related potentials (ERPs). The vMMN is measured as the difference between the ERPs to infrequent (deviant) and frequent (standard) stimuli irrelevant to the ongoing task. In the present study, we used human faces expressing different emotions as deviants and standards. In such studies, participants perform various tasks, so their attention is diverted from the vMMN-related stimuli. If such tasks vary in their attentional demand, they might influence the outcome of vMMN studies. In this study, we compared four kinds of frequently used tasks: (1) a tracking task that demanded continuous performance, (2) a detection task where the target stimuli appeared at any time, (3) a detection task where target stimuli appeared only in the inter-stimulus intervals, and (4) a task where target stimuli were members of the stimulus sequence. This fourth task elicited robust vMMN, while in the other three tasks, deviant stimuli elicited moderate posterior negativity (vMMN). We concluded that the ongoing task had a marked influence on vMMN; thus, it is important to consider this effect in vMMN studies.

Emotional expression visual mismatch negativity in children

Detection of changes in facial emotions is crucial to communicate and to rapidly process threats in the environment. This function develops throughout childhood via modulations of the earliest brain responses, such as the P100 and the N170 recorded using electroencephalography. Automatic brain signatures can be measured through expression-related visual mismatch negativity (vMMN), which reflects the processing of unattended changes. While increasing research has investigated vMMN processing in adults, few studies have been conducted on children. Here, a controlled paradigm previously validated was used to disentangle specific responses to emotional deviants (angry face) from that of neutral deviants. Latencies and amplitudes of P100 and N170 both decrease with age, confirming that sensory and face-specific activity is not yet mature in school-aged children. Automatic change detection-related activity is present in children, with a similar vMMN pattern in response to both emotional and neutral deviant stimuli to what previously observed in adults. However, vMMN processing is delayed in children compared to adults and no emotion-specific response is yet observed, suggesting nonmature automatic detection of salient emotional cues. To our knowledge, this is the first study investigating expression-related vMMN in school-aged children, and further investigations are needed to confirm these results.

Automatic change detection: Mismatch negativity and the now-classic Rensink, O’Reagan, and Clark (1997) stimuli

Change blindness experiments had demonstrated that detection of significant changes in natural images is extremely difficult when brief blank fields are placed between alternating displays of an original and a modified scene. On the other hand, research on the visual mismatch negativity (vMMN) component of the event-related potentials (ERPs) identified sensitivity to events (deviants) different from the regularity of stimulus sequences (standards), even if the deviant and standard events are non-attended. The present study sought to investigate the apparent controversy between the experience under the change blindness paradigm and the ERP results. To this end, the stimulus of Rensink, O’Reagen, and Clark (1997) was adapted to a passive oddball ERP paradigm to investigate the underlying processing differences between the standard (original) and deviant (altered) stimuli measured in 22 subjects. Posterior negativity within the 280–330 ms latency range emerged as the difference between ERPs elicited by standard and deviant stimuli, identified as visual mismatch negativity (vMMN). These results raise the possibility that change blindness is not based on the lack of detailed visual representations or the deficiency of comparing two representations. However, effective discrimination of the two scene versions requires considerable frequency differences between them.

Visual Mismatch Negativity Reflects Enhanced Response to the Deviant: Evidence From Event-Related Potentials and Electroencephalogram Time-Frequency Analysis

Automatic detection of information changes in the visual environment is crucial for individual survival. Researchers use the oddball paradigm to study the brain’s response to frequently presented (standard) stimuli and occasionally presented (deviant) stimuli. The component that can be observed in the difference wave is called visual mismatch negativity (vMMN), which is obtained by subtracting event-related potentials (ERPs) evoked by the deviant from ERPs evoked by the standard. There are three hypotheses to explain the vMMN. The sensory fatigue (or refractoriness) hypothesis considers that weakened neural activity caused by repetition results in decreased ERPs of the standard. The memory trace hypothesis proposes that vMMN results from increased responses to the deviant. The predictive coding hypothesis attributes the difference to enhanced responses for deviants and suppression for standards. However, when distinguishing between these effects, previous researchers did not consider the effect of low-level features on the vMMN. In this experiment, we used face sequences composed of different emotions (e.g., neutral and fearful face) and presented an oddball sequence, a reverse oddball sequence, and an equiprobable sequence to participants. The deviant of the oddball sequence was subtracted from the standard of the oddball sequence, the reverse oddball sequence, and the same type of stimulus of the equiprobable sequence to get oddball-vMMN (vMMN1), reverse oddball-vMMN (vMMN2), and equiprobable-vMMN (vMMN3), respectively. The results showed no significant difference between vMMN2 and vMMN3 in 100–350 ms following stimulus onset, while the vMMN effect was significant, indicating that the probability of the standard did not affect vMMN, which supported the memory trace hypothesis. Additionally, the fearful-related vMMN were more negative than the neutral-related vMMN within the range of 100–150 ms, suggesting a negative bias. We analyzed the source location of different vMMNs. There was no significant difference in brain regions between different vMMNs. Time-frequency analysis showed that the deviant had stronger theta-band oscillatory than the standard (visual mismatch oscillatory responses, vMORs). However, there was no difference between vMORs2 and vMORs3, indicating that vMORs reflect an enhanced response to the deviant in terms of neural oscillation, supporting the memory trace hypothesis.

Visual Mismatch Negativity: A Mini-Review of Non-pathological Studies With Special Populations and Stimuli

In this mini-review, we summarized the results of 12 visual mismatch negativity (vMMN) studies that attempted to use this component as a tool for investigating differences between non-clinical samples of participants as well as the possibility of automatic discrimination in the case of specific categories of visual stimuli. These studies investigated the effects of gender, the effects of long-term differences between the groups of participants (fitness, experience in different sports, and Internet addiction), and the effects of short-term states (mental fatigue and hypoxia), as well as the vMMN effect elicited by artworks as a special stimulus category.

Neural prediction errors depend on how an expectation was formed

When a visual event is unexpected, because it violates a train of repeated events, it excites a greater positive electrical potential at sensors positioned above occipital-parietal human brain regions (the P300). Such events can also seem to have an increased duration relative to repeated (implicitly expected) events. However, recent behavioural evidence suggests that when events are unexpected because they violate a declared prediction-a guess-there is an opposite impact on duration perception. The neural consequences of incorrect declared predictions have not been examined. We replicated the finding whereby repetition violating events elicit a larger P300 response. However, we found that events that violated a declared prediction entrained an opposite pattern of response-a smaller P300. These data suggest that the neural consequences of a violated prediction are not uniform but depend on how the prediction was formed.

Event-related potentials evidence for long-term audiovisual representations of phonemes in adults

The presence of long-term auditory representations for phonemes has been well-established. However, since speech perception is typically audiovisual, we hypothesized that long-term phoneme representations may also contain information on speakers' mouth shape during articulation. We used an audiovisual oddball paradigm in which, on each trial, participants saw a face and heard one of two vowels. One vowel occurred frequently (standard), while another occurred rarely (deviant). In one condition (neutral), the face had a closed, non-articulating mouth. In the other condition (audiovisual violation), the mouth shape matched the frequent vowel. Although in both conditions stimuli were audiovisual, we hypothesized that identical auditory changes would be perceived differently by participants. Namely, in the neutral condition, deviants violated only the audiovisual pattern specific to each block. By contrast, in the audiovisual violation condition, deviants additionally violated long-term representations for how a speaker's mouth looks during articulation. We compared the amplitude of mismatch negativity (MMN) and P3 components elicited by deviants in the two conditions. The MMN extended posteriorly over temporal and occipital sites even though deviants contained no visual changes, suggesting that deviants were perceived as interruptions in audiovisual, rather than auditory only, sequences. As predicted, deviants elicited larger MMN and P3 in the audiovisual violation compared to the neutral condition. The results suggest that long-term representations of phonemes are indeed audiovisual.

Older Adults Automatically Detect Age of Older Adults' Photographs: A Visual Mismatch Negativity Study

The human face is one of the most frequently used stimuli in vMMN (visual mismatch negativity) research. Previous studies showed that vMMN is sensitive to facial emotions and gender, but investigations of age-related vMMN differences are relatively rare. The aim of this study was to investigate whether the models' age in photographs were automatically detected, even if the photographs were not parts of the ongoing task. Furthermore, we investigated age-related differences, and the possibility of different sensitivity to photographs of participants' own versus different ages. We recorded event-related potentials (ERPs) to faces of young and old models in younger (N = 20; 18-30 years) and older groups (N = 20; 60-75 years). The faces appeared around the location of the field of a tracking task. In sequences the young or the old faces were either frequent (standards) or infrequent (deviants). According to the results, a regular sequence of models' age is automatically registered, and faces violating the models' age elicited the vMMN component. However, in this study vMMN emerged only in the older group to same-age deviants. This finding is explained by the less effective inhibition of irrelevant stimuli in the elderly, and corresponds to own-age bias effect of recognition studies.

Do categorical representations modulate early automatic visual processing? A visual mismatch-negativity study

Perceptual categorization is an important cognitive function. In the auditory domain, categorization already occurs within the first 200 ms of information processing, as indexed by the mismatch negativity. Here, we assessed the characteristics of the visual mismatch negativity (vMMN) elicited during the categorization of previously unknown visual stimuli. To examine this, we used five-dot patterns with characteristics that allow for the formation of categories through rotation and reflection but not through other physical properties. To assess whether or not between-category and within-category vMMN differ in amplitude, the data was analyzed with the Bayesian approach. We observed that both between-category and within-category deviants elicited a vMMN, but that both vMMNs were comparable in magnitude. This finding suggests that abstract categorical representations are not always automatically processed at early visual stages and demonstrates limitations of generalization from the auditory domain to visual domain.

Deficits in Visual Processing During Hypoxia as Evidenced by Visual Mismatch Negativity

INTRODUCTION: Hypoxia is an ever-present threat in tactical aviation and gained recent attention due to its putative role in physiological episodes. Previous work has demonstrated that hypoxia negatively impacts a variety of sensory, cognitive, and motor systems. In particular, the visual system is one of the earliest systems affected by hypoxia. While the majority of previous studies have relied on self-report and behavioral testing, the use of event-related potentials as a novel tool to monitor responses to low oxygen in humans has recently been investigated. Specifically, ERP components that are evoked passively in response to unattended changes in background sensory stimulation have been explored.METHOD: Subjects (N 28) completed a continuous visuomotor tracking task while EEG was recorded. During the tracking task, a series of standard color checkerboard patterns were presented in the periphery while occasionally a deviant color checkerboard was presented. The visual mismatch negativity (MMN) component was assessed in response to the deviant compared to the standard stimuli. Subjects completed two sessions in counterbalanced order that only differed by the oxygen concentration breathed (10.6% vs. 20.4%).RESULTS: Results demonstrated a significant reduction in the amplitude of the visual MMN under hypoxic compared to normoxic conditions, showing a 50% reduction in amplitude during hypoxia. Our results suggest that during low-oxygen exposure the ability to detect environmental changes and process sensory information is impaired.DISCUSSION: The visual MMN may represent an early and reliable predictor of sensory and cognitive deficits during hypoxia exposure, which may be of great use to the aviation community.Blacker KJ, Seech TR, Funke ME, Kinney MJ. Deficits in visual processing during hypoxia as evidenced by visual mismatch negativity. Aerosp Med Hum Perform. 2021; 92(5):326332.

Impaired Audiovisual Representation of Phonemes in Children with Developmental Language Disorder

We examined whether children with developmental language disorder (DLD) differed from their peers with typical development (TD) in the degree to which they encode information about a talker’s mouth shape into long-term phonemic representations. Children watched a talker’s face and listened to rare changes from [i] to [u] or the reverse. In the neutral condition, the talker’s face had a closed mouth throughout. In the audiovisual violation condition, the mouth shape always matched the frequent vowel, even when the rare vowel was played. We hypothesized that in the neutral condition no long-term audiovisual memory traces for speech sounds would be activated. Therefore, the neural response elicited by deviants would reflect only a violation of the observed audiovisual sequence. In contrast, we expected that in the audiovisual violation condition, a long-term memory trace for the speech sound/lip configuration typical for the frequent vowel would be activated. In this condition then, the neural response elicited by rare sound changes would reflect a violation of not only observed audiovisual patterns but also of a long-term memory representation for how a given vowel looks when articulated. Children pressed a response button whenever they saw a talker’s face assume a silly expression. We found that in children with TD, rare auditory changes produced a significant mismatch negativity (MMN) event-related potential (ERP) component over the posterior scalp in the audiovisual violation condition but not in the neutral condition. In children with DLD, no MMN was present in either condition. Rare vowel changes elicited a significant P3 in both groups and conditions, indicating that all children noticed auditory changes. Our results suggest that children with TD, but not children with DLD, incorporate visual information into long-term phonemic representations and detect violations in audiovisual phonemic congruency even when they perform a task that is unrelated to phonemic processing.

Magnetoencephalography Responses to Unpredictable and Predictable Rare Somatosensory Stimuli in Healthy Adult Humans

Mismatch brain responses to unpredicted rare stimuli are suggested to be a neural indicator of prediction error, but this has rarely been studied in the somatosensory modality. Here, we investigated how the brain responds to unpredictable and predictable rare events. Magnetoencephalography responses were measured in adults frequently presented with somatosensory stimuli (FRE) that were occasionally replaced by two consecutively presented rare stimuli [unpredictable rare stimulus (UR) and predictable rare stimulus (PR); p = 0.1 for each]. The FRE and PR were electrical stimulations administered to either the little finger or the forefinger in a counterbalanced manner between the two conditions. The UR was a simultaneous electrical stimulation to both the forefinger and the little finger (for a smaller subgroup, the UR and FRE were counterbalanced for the stimulus properties). The grand-averaged responses were characterized by two main components: one at 30-100 ms (M55) and the other at 130-230 ms (M150) latency. Source-level analysis was conducted for the primary somatosensory cortex (SI) and the secondary somatosensory cortex (SII). The M55 responses were larger for the UR and PR than for the FRE in both the SI and the SII areas and were larger for the UR than for the PR. For M150, both investigated areas showed increased activity for the UR and the PR compared to the FRE. Interestingly, although the UR was larger in stimulus energy (stimulation of two fingers at the same time) and had a larger prediction error potential than the PR, the M150 responses to these two rare stimuli did not differ in source strength in either the SI or the SII area. The results suggest that M55, but not M150, can possibly be associated with prediction error signals. These findings highlight the need for disentangling prediction error and rareness-related effects in future studies investigating prediction error signals.

Do categorical representations modulate early perceptual or later cognitive visual processing? An ERP study

Encoding of perceptual categorical information has been observed in later cognitive processing like memory encoding and maintenance, starting around 300 ms after stimulus onset (P300). However, it remains open whether categorical information is also encoded in early perceptual processing steps (reflected in the mismatch negativity component; vMMN). The main goal of this study was to assess the influence of categorical information on both early perceptual (i.e., vMMN component) and later cognitive (i.e., P300 component) processing within one paradigm. Hence, we combined an oddball paradigm with a delayed memory task. We used five-dot patterns belonging to different categories even though categorical information is not mirrored in their physical characteristics. Distinct vMMNs were observed for patterns belonging to the same as compared to different categories, suggesting that abstract categorical information was encoded during early perceptual processing. However, inconsistent with prior studies, we observed no effect of categories on the P300, indicating no additional encoding of categorical information in later cognitive stages of processing. Our findings emphasize that the encoding of categorical information depends on specific task demands and hence is more flexible and dynamic than previously suggested.

Automatic detection of peripheral stimuli in shooters and handball players: an event-related potential study

This study examined the practice-related sensitivity of automatic change detection. The visual mismatch negativity (vMMN) component of event-related potentials was compared in handball players and in sport shooters. Whereas effective performance in handball requires processing of a wide visual field, effective performance in shooting requires concentration to a narrow field. Thus, we hypothesized larger sensitivity to peripheral stimuli violating the regularity of sequential stimulation in handball players. Participants performed a tracking task, while task-irrelevant checkerboard patterns (a frequent and an infrequent type) were presented in the lateral parts of the visual field. We analyzed the vMMN, a signature of automatic detection of violating sequential regularity, and sensory components (P1, N1, and P2). We obtained larger vMMN in the handball players' group indicating larger sensitivity to peripheral stimuli. These results suggest the plasticity of the automatic visual processing, i.e., it can adapt to sport-specific demands, and this can be captured even in a short experimental session in the laboratory.

Emotional visual mismatch negativity: a joint investigation of social and non-social dimensions in adults with autism

Unusual behaviors and brain activity to socio-emotional stimuli have been reported in Autism Spectrum Disorder (ASD). Atypical reactivity to change and intolerance of uncertainty are also present, but little is known on their possible impact on facial expression processing in autism. The visual mismatch negativity (vMMN) is an electrophysiological response automatically elicited by changing events such as deviant emotional faces presented among regular neutral faces. While vMMN has been found altered in ASD in response to low-level changes in simple stimuli, no study has investigated this response to visual social stimuli. Here two deviant expressions were presented, neutral and angry, embedded in a sequence of repetitive neutral stimuli. vMMN peak analyses were performed for latency and amplitude in early and late time windows. The ASD group presented smaller amplitude of the late vMMN to both neutral and emotional deviants compared to the typically developed adults (TD) group, and only the TD group presented a sustained activity related to emotional change (i.e., angry deviant). Source reconstruction of the vMMNs further revealed that any change processing elicited a reduced activity in ASD group compared to TD in the saliency network, while the specific processing emotional change elicited activity in the temporal region and in the insula. This study confirms atypical change processing in ASD and points to a specific difficulty in the processing of emotional changes, potentially playing a crucial role in social interaction deficits. Nevertheless, these results require to be further replicated with a greater sample size and generalized to other emotional expressions.

Visual mismatch responses index surprise signalling but not expectation suppression

The ability to distinguish between commonplace and unusual sensory events is critical for efficient learning and adaptive behaviour. This has been investigated using oddball designs in which sequences of often-appearing (i.e., expected) stimuli are interspersed with rare (i.e., surprising) deviants. Resulting differences in electrophysiological responses following surprising compared to expected stimuli are known as visual mismatch responses (VMRs). VMRs are thought to index co-occurring contributions of stimulus repetition effects, expectation suppression (that occurs when one's expectations are fulfilled), and expectation violation (i.e., surprise) responses; however, these different effects have been conflated in existing oddball designs. To better isolate and quantify effects of expectation suppression and surprise, we adapted an oddball design based on Fast Periodic Visual Stimulation (FPVS) that controls for stimulus repetition effects. We recorded electroencephalography (EEG) while participants (N = 48) viewed stimulation sequences in which a single face identity was periodically presented at 6 Hz. Critically, one of two different face identities (termed oddballs) appeared as every 7th image throughout the sequence. The presentation probabilities of each oddball image within a sequence varied between 10 and 90%, such that participants could form expectations about which oddball face identity was more likely to appear within each sequence. We also included 'expectation neutral' 50% probability sequences, whereby consistently biased expectations would not be formed for either oddball face identity. We found that VMRs indexed surprise responses, and effects of expectation suppression were absent. That is, ERPs were more negative-going at occipitoparietal electrodes for surprising compared to neutral oddballs, but did not differ between expected and neutral oddballs. Surprising oddball-evoked ERPs were also highly similar across the 10-40% appearance probability conditions. Our findings indicate that VMRs which are not accounted for by repetition effects are best described as an all-or-none surprise response, rather than a minimisation of prediction error responses associated with expectation suppression.

Temporal attention boosts perceptual effects of spatial attention and feature-based attention

Temporal attention, that is, the process of anticipating the occurrence of a stimulus at a given time point, has been shown to improve perceptual processing of visual stimuli. In the present study, we investigated whether and how temporal attention interacts with spatial attention and feature-based attention in visual selection. To monitor the influence of the three different attention dimensions on perceptual processing, we measured event-related potentials (ERPs). Our participants performed a visual search task, in which a colored singleton was presented amongst homogenous distractors. We manipulated spatial and feature-based attention by requiring participants to respond only to target singletons in a particular color and at a to-be-attended spatial location. We manipulated temporal attention by means of an explicit temporal cue that announced either validly or invalidly the occurrence of the search display. We obtained early ERP effects of spatial attention and feature-based attention at the validly cued but not at the invalidly cued time point. Taken together, our results suggest that temporal attention boosts early effects of spatial and feature-based attention.

Pearls and pitfalls in brain functional analysis by event-related potentials: a narrative review by the Italian Psychophysiology and Cognitive Neuroscience Society on methodological limits and clinical reliability-part I

Event-related potentials (ERPs) are obtained from the electroencephalogram (EEG) or the magnetoencephalogram (MEG, event-related fields (ERF)), extracting the activity that is time-locked to an event. Despite the potential utility of ERP/ERF in cognitive domain, the clinical standardization of their use is presently undefined for most of procedures. The aim of the present review is to establish limits and reliability of ERP medical application, summarize main methodological issues, and present evidence of clinical application and future improvement. The present section of the review focuses on well-standardized ERP methods, including P300, Contingent Negative Variation (CNV), Mismatch Negativity (MMN), and N400, with a chapter dedicated to laser-evoked potentials (LEPs). One section is dedicated to proactive preparatory brain activity as the Bereitschaftspotential and the prefrontal negativity (BP and pN). The P300 and the MMN potentials have a limited but recognized role in the diagnosis of cognitive impairment and consciousness disorders. LEPs have a well-documented usefulness in the diagnosis of neuropathic pain, with low application in clinical assessment of psychophysiological basis of pain. The other ERP components mentioned here, though largely applied in normal and pathological cases and well standardized, are still confined to the research field. CNV, BP, and pN deserve to be largely tested in movement disorders, just to explain possible functional changes in motor preparation circuits subtending different clinical pictures and responses to treatments.

Consciousness modulates the automatic change detection of masked emotional faces: Evidence from visual mismatch negativity

The ability to automatically detect changes in the environment is crucial for organisms to survive. In the visual system, changes in visual stimuli may evoke visual mismatch negativity (vMMN), an event-related potential (ERP) component that reflects automatic change detection. Previous studies that used visual masking to examine the effects of consciousness did not yield evidence that vMMN could be elicited by subliminal stimuli. However, these studies used relatively simple visual features. To further examine the role of consciousness in vMMN, the present study used emotional (happy and fearful) faces, which are biologically and socially significant visual stimuli. A passive oddball paradigm was employed, and we found that only fearful faces could evoke vMMN at a low consciousness level. Furthermore, the fear-related vMMN was enhanced by the consciousness level, and localized in regions of the brain associated with emotional face processing and the prefrontal cortex. We also found that the emotional visual mismatch oscillatory responses (vMORs) were associated with the enhancement of the alpha-band oscillation. Moreover, consciousness could weaken the happiness-related vMOR. These results suggested that changes to emotional faces-especially fearful faces-could be unconsciously detected by the brain. More importantly, this automatic change detection mechanism could be modulated by consciousness.

The effect of hand motion and object orientation on the automatic detection of orientation: A visual mismatch negativity study

We investigated the effects of voluntary hand movements and continuously present objects on the automatic detection of deviant stimuli in a passive oddball paradigm. The visual mismatch negativity (vMMN) component of event-related potentials (ERPs) was measured as the index of automatic deviant detection. The stimuli were textures consisting of parallel, oblique bars with frequent (standard) and infrequent (deviant) orientation. Traditional vMMN was measured by the difference between ERPs to frequent (standard) and infrequent (deviant) textures. Additionally, we measured 'genuine' vMMN by comparing the ERPs to deviant and control textures in the equal probability procedure. Compatible and incompatible hand movement directions to the standard texture had no influence on 'traditional' vMMN and elicited no 'genuine' vMMN. However, the deviant texture elicited 'genuine' vMMN if the orientation of a continuously present rectangle was different from the standard (and identical to the deviant) texture orientation. Our results suggest that the direction of voluntary hand movement and the orientation of task-irrelevant visual patterns do not acquire common memory representation, but a continuously present object contributes to the detection of sequential regularity violation.

Stimulus probability affects the visual N700 component of the event-related potential

OBJECTIVE

To examine whether the occipito-temporal visual N700 component of the event-related potential is sensitive to stimulus probabilities.

METHODS

P1, N1, P3, and, in particular, the occipito-temporal N700 component of the event-related potential were analysed in response to frequent and rare non-target letters of a continuous performance task in 200 healthy adolescents. Additionally, amplitude habituation with time was examined for the occipito-temporal N700 and N1 components.

RESULTS

The visual P1, N1, and occipito-temporal N700 components were significantly larger in response to rare letters than to frequent letters, whereas the P3 component demonstrated no amplitude difference. Over time, the occipito-temporal N700 amplitude decreased in response to the rare letters, while the N1 amplitude increased, to both, frequent and rare letters.

CONCLUSIONS

This study provides first evidence that the visual occipito-temporal N700 is sensitive to stimulus probabilities, suggesting an enhanced post-processing of rare stimuli in secondary visual areas. The distinct habituation patterns of occipito-temporal N700 and N1 amplitudes distinguish repetition effects on stimulus post-processing (N700) from those on perception (N1).

SIGNIFICANCE

The enhanced N700 component to rare stimuli might reflect an orienting response and underlying attentional processes. The N700 sensitivity to stimulus probabilities should be examined in patient groups with attentional deficits.

Dysfunction in Automatic Processing of Emotional Facial Expressions in Patients with Obstructive Sleep Apnea Syndrome: An Event-Related Potential Study

AIM

Obstructive sleep apnea syndrome (OSAS) is a prevalent chronic disease characterized by sleep fragmentation and intermittent hypoxemia. Several studies suggested that electrophysiological changes and neurocognitive abnormalities occurred in OSAS patients. In this study, we compared automatic processing of emotional facial expressions schematic in OSAS patients and matched healthy controls via assessing expression-related mismatch negativity (EMMN).

METHODS

Twenty-two OSAS patients (mean age 44.59 years) and twenty-one healthy controls (mean age 42.71 years) were enrolled in this study. All participants underwent Montreal Cognitive Assessment (MoCA) scale test and polysomnographic recording. An expression-related oddball paradigm was used to elicit EMMN and the electroencephalogram was recorded and analyzed. Furthermore, Pearson's correlations were calculated to discuss the correlation between neuropsychological test scores, clinical variables and electrophysiological data.

RESULTS

Compared with healthy controls, OSAS sufferers demonstrated significantly reduced EMMN mean amplitudes within corresponding time intervals, regardless of happy or sad conditions. Meanwhile, we observed that amplitude of sad EMMN was larger (more negative) than happy EMNN in healthy controls, while not in patients. Moderate correlations were found between MoCA test scores, sleep parameters and EMMN amplitudes.

CONCLUSION

Our findings suggested pre-attentive dysfunction of processing emotional facial expressions in patients with OSAS, without the existence of negative bias effect. Moreover, correlation analysis showed that clinical characteristics of OSAS patients could affect EMMN amplitudes. Further studies on the advantages of EMMN as clinical and electrophysiological indicators of OSAS are warranted.

Automatic Change Detection of Emotional and Neutral Body Expressions: Evidence From Visual Mismatch Negativity

Rapidly and effectively detecting emotions in others is an important social skill. Since emotions expressed by the face are relatively easy to fake or hide, we often use body language to gauge the genuine emotional state of others. Recent studies suggest that expression-related visual mismatch negativity (vMMN) reflects the automatic processing of emotional changes in facial expression; however, the automatic processing of changes in body expression has not yet been studied systematically. The current study uses an oddball paradigm where neutral body actions served as standard stimuli, while fearful body expressions and other neutral body actions served as two different deviants to define body-related vMMN, and to compare the mechanisms underlying the processing of emotional changes to neutral postural changes. The results show a more negative vMMN amplitude for fear deviants 210-260 ms after stimulus onset which corresponds with the negativity bias that was obtained on the N190 component. In earlier time windows, the vMMN amplitude following the two types of deviant stimuli are identical. Therefore, we present a two-stage model for processing changes in body posture, where changes in body posture are processed in the first 170-210 ms, but emotional changes in the time window of 210-260 ms.

Action-induced adjustment of prediction explains no visual mismatch negativity to self-generated deviants

Visual mismatch negativity (VMMN) is an event-related brain potential component elicited by infrequent deviant events embedded among frequent standard events in a visual stimulus sequence. Although VMMN is known to be elicited by deviant events in a stimulus-driven manner, it was recently shown that VMMN can be drastically affected by the participant's voluntary action. With a paradigm in which participants were required to press one button frequently (about 90%) and another button infrequently (10%) in random order, and produce a visual stimulus sequence consisting of deviant and standard events, VMMN was elicited by deviant events triggered by the frequently-performed button press that should generate a standard event, but not by deviant events triggered by the infrequently-performed button press that should generate a corresponding deviant event. The present study replicated these previous findings and further demonstrated that VMMN was elicited by deviant events triggered by the infrequently-performed button press that should generate a different deviant event. These results support the hypothesis that VMMN-generating processes can be adjusted according to the participant's voluntary action, and rule out an alternative hypothesis that VMMN-generating processes are terminated when participants change their action. This adjustment is thought to be implemented so that self-generated deviants that carry no novel information are discarded, while externally-generated deviants that may carry novel information are selectively detected.

Visual mismatch negativity and stimulus-specific adaptation: the role of stimulus complexity

The present study investigated the function of the brain activity underlying the visual mismatch negativity (vMMN) event-related potential (ERP) component. Snowflake patterns (complex stimuli) were presented as deviants and oblique bar patterns (simple stimuli) as standards, and vice versa in a passive oddball paradigm. Control (equiprobable) sequences of either complex shape patterns or oblique bar patterns with various orientations were also presented. VMMN appeared as the difference between the ERP to the oddball deviant and the ERP to the control (deviant minus control ERP difference). Apart from the shorter latency of the vMMN to the oblique bar pattern as deviant, vMMN to both deviants was similar, i.e., there was no amplitude difference. We attributed the function of the brain processes underlying vMMN to the detection of the infrequent stimulus type (also represented in memory) instead of a call for further processing (a possibility for acquiring more precise representation) of the deviant. An unexpected larger adaptation (control minus standard ERP difference) to the snowflake pattern was also obtained. We suggest that this was due to the acquisition of a more elaborate memory representation of the more complex stimulus.

Examining the Role of Spatial Changes in Bimodal and Uni-Modal To-Be-Ignored Stimuli and How They Affect Short-Term Memory Processes

This study examines the potential vulnerability of short-term memory processes to distraction by spatial changes within to-be-ignored bimodal, vibratory, and auditory stimuli. Participants were asked to recall sequences of serially presented digits or locations of dots while being exposed to to-be-ignored stimuli. On unexpected occasions, the bimodal to-be-ignored sequence, vibratory to-be-ignored sequence, or auditory to-be-ignored sequence changed their spatial origin from one side of the body (e.g., ear and arm, arm only, ear only) to the other. It was expected that the bimodal stimuli would make the spatial change more salient compared to that of the uni-modal stimuli and that this, in turn, would yield an increase in distraction of serial short-term memory in both the verbal and spatial domains. Our results support this assumption as a disruptive effect of the spatial deviant was only observed when presented within the bimodal to-be-ignored sequence: uni-modal to-be-ignored sequences, whether vibratory: or auditory, had no impact on either verbal or spatial short-term memory. Implications for models of attention capture and the potential special attention capturing role of bimodal stimuli are discussed.

A neuromarker of clinical outcome in attention bias modification therapy for social anxiety disorder

BACKGROUND

Attention bias modification (ABM) therapy aims to modify threat-related attention patterns via computerized tasks. Despite showing medium clinical effect sizes for anxiety disorders, underlying neural-cognitive mechanisms of change remain unclear. We used visual mismatch negativity (vMMN), an event-related potential sensitive to violations of learned statistical contingencies, to assess therapy-related contingency extraction processes in healthy participants and in patients with social anxiety disorder (SAD). We then assessed whether vMMN amplitude predicts ABM treatment outcome.

METHODS

A modified version of the dot-probe task was used to elicit vMMN, in which 80% of trials were standard and 20% were deviant. In study 1, 30 healthy adults were randomly assigned to one of two ABM conditions: one in which threat-congruent targets were deviant trials and threat-incongruent targets were standard trials, and another in which the contingency was reversed. Electroencephalography (EEG) was continuously measured and vMMN analyzed. In study 2, 38 patients with SAD underwent six sessions of ABM therapy. We tested whether rule extraction in the ABM task, indicated by vMMN amplitude, predicts treatment outcome.

RESULTS

vMMN clearly emerged over prespecified scalp locations indicating contingency extraction during ABM (study 1). vMMN amplitude predicted clinical improvement after ABM therapy, uniquely accounting for 7% and 14.4% of the variance in clinician-rated and self-reported posttreatment SAD symptoms, respectively.

CONCLUSIONS

vMMN emerges as a neural marker for contingency learning in ABM, suggesting a significant role for contingency extraction processes in the clinical efficacy of this therapy.

Visual mismatch negativity and representational momentum: Their possible involvement in the same automatic prediction

To maintain real-time interaction with a dynamically changing visual object, the brain is thought to automatically predict the next state of the object based on the pattern of its preceding changes. A behavioral phenomenon known as representational momentum (RM: forward displacement of the remembered final state of an object along its preceding change pattern) and an electrophysiological phenomenon known as visual mismatch negativity (VMMN: an event-related brain potential component that is elicited when an object suddenly deviates from its preceding change pattern) have each indicated the existence of such automatic predictive processes. However, there has been no direct investigation of whether or not these phenomena are involved in the same predictive processes. To address this issue, the present study examined the correlation between RM and VMMN by using a hybrid paradigm in which both phenomena can be measured for the rotation of a bar. The results showed that the magnitudes of RM and VMMN were positively correlated; participants who exhibited greater RM along the regular rotation of a bar tended to show greater VMMN in response to sudden reversal embedded in the regular rotation of a bar. This result provides empirical support for the hypothesis that RM and VMMN may be involved in the same automatic predictive processes. Due to the methodological limitations of a correlation analysis, this hypothesis has to be carefully tested in future studies that examine the relationship between RM and VMMN from multiple perspectives.

Sex and Physiological Cycles Affect the Automatic Perception of Attractive Opposite-Sex Faces: A Visual Mismatch Negativity Study

Facial attractiveness plays important roles in social interaction. Electrophysiological and neuroimaging studies found several brain areas to be differentially responsive to attractive relative to unattractive faces. However, little is known about the time course of the information processing, especially under the unattended condition. Based on a "cross-modal delayed response" paradigm, the present study aimed to explore the automatic mechanism of facial attractiveness processing of females with different physiological cycles and males, respectively, through recording the event-related potentials in response to (un)attractive opposite-sex faces by two experiments. The attractiveness-related visual mismatch negativity (attractiveness vMMN) in posterior scalp distribution was recorded in both the experiments, which indicated that attractive faces could be processed automatically. And high-attractive opposite-sex faces can elicit larger vMMN in males than females in menstrual period in Study 1, but similar as females in ovulatory period in Study 2. Furthermore, by comparison, the latency of attractiveness vMMN in females with the ovulatory period was the longest. These results indicated as follows: (1) Males were more sensitive to attractive female faces, (2) females in ovulatory period were also attracted by the attractive male faces, (3) the long vMMN latency in females during ovulatory period suggested a special reproductive motivation to avoid being tainted by genes, which takes priority over the breeding motivation.

Automatic change detection and spatial attention: a visual mismatch negativity study

Visual mismatch negativity (vMMN) is the electrophysiological correlate of automatic detection of unattended changes in the visual environment. However, vMMNs' relatedness to spatial attention has not been explicitly tested. Thus, the aim of the study was to investigate the effects of spatial attention on the vMMN event-related potential component. To this end, participants were instructed to fixate and attend to task-related stimuli. In an oddball sequence, offset stimuli were applied, i.e., from time-to time, the two sides of permanently presented objects disappeared. Distance between the task-related and unrelated events resulted in the typical finding of spatial attention; the amplitude of the N1 component was larger at the shorter distance between the two kinds of events. VMMN was elicited by the deviant vanishing parts, with no reliable effect of distance between the task-field and vMMN-related stimuli. In terms of the difference potentials, vMMN was followed by a positive posterior component in the 270-330 ms range. This positivity was much larger when the task-field was close to vMMN-related stimuli. The reappearance of the vanishing parts was also investigated. The reappearance of the whole objects after a deviant offset elicited vMMN but only when the task-field was close to the oddball sequence. We concluded that infrequently vanishing parts of objects are detected automatically. However, these deviant events initiate orientation only if the objects are close to the field of task-relevant events. Similarly, automatic registration of the rare but expected events are registered only in the visual field close to the focus of attention.

At a Single Glance: Fast Periodic Visual Stimulation Uncovers the Spatio-Temporal Dynamics of Brief Facial Expression Changes in the Human Brain

Detecting brief changes of facial expression is vital for social communication. Yet, how reliably, how fast these changes are detected and how long they are processed in the human brain remain unknown. High-density electroencephalogram (EEG) was recorded in 18 participants presented with a neutral-expression face at a rate of 5.88 Hz (F) for 80 s. Every five faces, the face changed expression to fear, disgust or happiness (different stimulation sequences). The resulting 1.18 Hz (F/5) EEG response and its harmonics objectively indexed detection of a brief change of facial expression. This response was recorded in every participant in a few minutes but was largely reduced for inverted faces, indicating that it reflects high-level processes. Although this response focused on occipito-temporal sites, different expression changes evoked reliably distinct topographical maps, pointing to partly distinct neural generators. These effects were also observed at a faster 12 Hz frequency rate and a lower ratio of expression change (1/9). Time-domain analysis showed that a brief change of expression inserted in a dynamic stimulation sequence elicits specific occipito-temporal responses between 100 and 310 ms, indicating a rapid change detection process followed by a long integration period of facial expression information in the human brain.

Automatic Change Detection in Older and Younger Women: A Visual Mismatch Negativity Study

BACKGROUND

In comparison to controlled (attentional) processing, relatively little is known about the age-related changes of the earlier (preattentive) processes. An event-related potential (ERP) index of preattentive (automatic) visual processing, the visual mismatch negativity (vMMN) is a good candidate for analyzing age-related differences in the automatic processing of visual events.

OBJECTIVE

So far results concerning age-related changes in vMMN have been equivocal. Our aim was to develop a method resulting in a reliable vMMN in a paradigm short enough to use in the applied field.

METHODS

We investigated an older (mean age: 66.4 years, n = 15) and a younger (mean age: 22.4 years, n = 15) group of healthy women. ERPs were obtained for checkerboard onset patterns in a passive oddball condition (during which participants performed a tracking task). One of the checkerboards was frequent (standard; p = 0.8), and the other was rare (deviant; p = 0.2).

RESULTS

vMMN emerged over posterior locations in the latency range of 100-300 ms in both age groups. The amplitude of the earlier part of the vMMN was similar in the older and the younger participants, but latency was longer in the older group. The later part of the vMMN was slightly diminished in the elderly.

CONCLUSION

Automatic detection of violated sequential regularities, reflected by the vMMN, emerged in the two age groups (earlier vMMN). However, detection of stimulus change, a preattentive visual process delayed in the elderly, and identification of the specific change was compromised in the older participants.

Visual mismatch negativity is unaffected by top-down prediction of the timing of deviant events

Visual mismatch negativity (VMMN) is an event-related brain potential component that is automatically elicited by infrequent (deviant) stimuli that are inserted among frequent (standard) stimuli (i.e., an oddball sequence). Although the elicitation of VMMN is basically determined in a stimulus-driven manner, it can be modulated by top-down control. In a previous study using a "patterned" oddball sequence, where deviant (D) stimuli were regularly inserted among standard (S) stimuli (i.e., repetitions of an SSSSD pattern), VMMN was largely reduced when participants noticed the SSSSD pattern and actively predicted both the identity and timing of the deviant stimuli compared to when they did not notice the SSSSD pattern and did not form such active prediction. The present study further investigated whether or not active prediction of only the timing of deviant stimuli is sufficient for the reduction of VMMN. With the patterned oddball sequence with one deviant (here, deviant stimuli were fixed throughout the block), VMMN was reduced when the participants noticed the SSSSD pattern and actively predicted both the identity and timing of deviant stimuli (i.e., replication of the previous finding). In contrast, with the patterned oddball sequence with two deviants (deviant stimuli were randomly varied between two possibilities), VMMN was not significantly reduced when the participants noticed the SSSSD pattern and actively predicted only the timing of deviant stimuli. These results suggest that active prediction of only the timing of deviant stimuli is not sufficient to reduce VMMN.

Visual mismatch negativity to vanishing parts of objects in younger and older adults

We investigated visual mismatch negativity (vMMN) to vanishing parts of continuously present objects by comparing the event-related potentials (ERPs) to infrequently (deviant) and frequently (standard) disappearing parts of the objects. This paradigm both excludes low-level stimulus-specific adaptation differences between the responses to deviants and standards, and increases the ecological validity of the stimuli. In comparison to frequently disappearing parts of the stimulus objects, infrequently vanishing parts elicited posterior negative event-related brain activity (vMMN). However, no vMMN emerged to the reappearance of the same parts of the objects. We compared the ERPs of an older and a younger sample of participants. In the 120-180 ms time period vMMN was similar in the two age groups, but in the 180-220 ms time period vMMN emerged only in the younger participants. We consider this difference as an index of more elaborate automatic processing of infrequent stimulus changes in younger adults.