fMRI reveals that involuntary visual deviance processing is resource limited

A multimodal cortical network of sensory expectation violation revealed by fMRI

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

Visual perceptual load and processing of somatosensory stimuli in primary and secondary somatosensory cortices

Load theory assumes that neural activation to distractors in early sensory cortices is modulated by the perceptual load of a main task, regardless of whether task and distractor share the same sensory modality or not. While several studies have investigated the question of load effects on distractor processing in early sensory areas, there is no functional magnetic resonance imaging (fMRI) study regarding load effects on somatosensory stimuli. Here, we used fMRI to investigate effects of visual perceptual load on neural responses to somatosensory stimuli applied to the wrist in a study with 44 participants. Perceptual load was manipulated by an established sustained visual detection task, which avoided simultaneous target and distractor presentations. Load was operationalized by detection difficulty of subtle or clear color changes of one of 12 rotating dots. While all somatosensory stimuli led to activation in somatosensory areas SI and SII, we found no statistically significant difference in brain activation to these stimuli under high compared to low sustained visual load. Moreover, exploratory Bayesian analyses supported the absence of differences. Thus, our findings suggest a resistance of somatosensory processing to at least some forms of visual perceptual load, possibly due to behavioural relevance of discrete somatosensory stimuli and separable attentional resources for the somatosensory and visual modality.

Fronto-occipital mismatch responses in pre-attentive detection of visual changes: Implication on a generic brain network underlying Mismatch Negativity (MMN)

Current theories of pre-attentive change detection suggest a regularity or prediction violation mechanism involving a frontotemporal network. Modulations of the early inferior frontal cortex (IFC) mismatch response representing the effort in comparing a stimulus to the prediction, the superior temporal cortex (STC) response indicating deviance detection, and the late IFC response representing prediction model updating were consistently demonstrated in auditory change detection using event-related optical signal (EROS). If the prediction violation hypothesis is universal, a generic neural mechanism should be found in all sensory modalities. We postulated a generic fronto-sensory cortical network underlying the prediction violation mechanism: the IFC is responsible for non-modality-specific prediction processes while the sensory cortices are responsible for modality-specific error signal generation process. This study examined the involvement of the IFC-occipital cortex (OC) network in visual pre-attentive change detection. The EROS mismatch responses to deviant bar arrays violating a fixed orientation regularity (low in regularity abstractness) were compared to that of deviant violating a rotational orientation regularity (high in abstractness) while the information available for establishing the prediction model was manipulated by varying the number of standards preceding the deviants. Modulations of the IFCOC mismatch response patterns by abstractness and train length reflected the processing demands on the prediction processes and were similar to that of the IFC-STC network in auditory change detection. These findings demonstrated that the fronto-sensory cortical network is not unique to auditory pre-attentive change detection and provided supports for a universal neural mechanism across sensory modalities as suggested by the prediction violation hypothesis.

EEG-based visual deviance detection in freely behaving mice

The mouse is widely used as an experimental model to study visual processing. To probe how the visual system detects changes in the environment, functional paradigms in freely behaving mice are strongly needed. We developed and validated the first EEG-based method to investigate visual deviance detection in freely behaving mice. Mice with EEG implants were exposed to a visual deviant detection paradigm that involved changes in light intensity as standard and deviant stimuli. By subtracting the standard from the deviant evoked waveform, deviant detection was evident as bi-phasic negativity (starting around 70 ms) in the difference waveform. Additionally, deviance-associated evoked (beta/gamma) and induced (gamma) oscillatory responses were found. We showed that the results were stimulus-independent by applying a "flip-flop" design and the results showed good repeatability in an independent measurement. Together, we put forward a validated, easy-to-use paradigm to measure visual deviance processing in freely behaving mice.

Automatic change detection and spatial attention: a visual mismatch negativity study

Visual mismatch negativity (vMMN) is the electrophysiological correlate of automatic detection of unattended changes in the visual environment. However, vMMNs' relatedness to spatial attention has not been explicitly tested. Thus, the aim of the study was to investigate the effects of spatial attention on the vMMN event-related potential component. To this end, participants were instructed to fixate and attend to task-related stimuli. In an oddball sequence, offset stimuli were applied, i.e., from time-to time, the two sides of permanently presented objects disappeared. Distance between the task-related and unrelated events resulted in the typical finding of spatial attention; the amplitude of the N1 component was larger at the shorter distance between the two kinds of events. VMMN was elicited by the deviant vanishing parts, with no reliable effect of distance between the task-field and vMMN-related stimuli. In terms of the difference potentials, vMMN was followed by a positive posterior component in the 270-330 ms range. This positivity was much larger when the task-field was close to vMMN-related stimuli. The reappearance of the vanishing parts was also investigated. The reappearance of the whole objects after a deviant offset elicited vMMN but only when the task-field was close to the oddball sequence. We concluded that infrequently vanishing parts of objects are detected automatically. However, these deviant events initiate orientation only if the objects are close to the field of task-relevant events. Similarly, automatic registration of the rare but expected events are registered only in the visual field close to the focus of attention.

Reduction in the Reverse-Bias Effect by an Abrupt Break in the Sequential Regularity of Visual Events

A bistable image is more likely to be initially perceived as the reversal of its preceding unambiguous version presented for a prolonged period. This perceptual bias is called the reverse-bias effect. We hypothesized that an abrupt break in the sequential regularity of visual events, synchronized with the onset of a bistable image, counteracts the reverse-bias effect in a similar manner to the disturbing effect of noise in the perceptual process. Under the condition in which the reverse-bias effect was achieved with the Necker lattice, the orientation of the bars around the lattice was simultaneously changed at the onset of the lattice, yielding an abrupt break in the sequential regularity of visual events besides the lattice. The results obtained showed that the reverse-bias effect was significantly reduced by the abrupt break, suggesting that an abrupt break in the sequential regularity of visual events perturbs the perceptual bias of the bistable image, similar to that caused by noise.

Automatic Change Detection to Facial Expressions in Adolescents: Evidence from Visual Mismatch Negativity Responses

Adolescence is a critical period for the neurodevelopment of social-emotional processing, wherein the automatic detection of changes in facial expressions is crucial for the development of interpersonal communication. Two groups of participants (an adolescent group and an adult group) were recruited to complete an emotional oddball task featuring on happy and one fearful condition. The measurement of event-related potential was carried out via electroencephalography and electrooculography recording, to detect visual mismatch negativity (vMMN) with regard to the automatic detection of changes in facial expressions between the two age groups. The current findings demonstrated that the adolescent group featured more negative vMMN amplitudes than the adult group in the fronto-central region during the 120–200 ms interval. During the time window of 370–450 ms, only the adult group showed better automatic processing on fearful faces than happy faces. The present study indicated that adolescent’s posses stronger automatic detection of changes in emotional expression relative to adults, and sheds light on the neurodevelopment of automatic processes concerning social-emotional information.

Electrophysiological correlates of self-specific prediction errors in the human brain

Recognising one's self, vs. others, is a key component of self-awareness, crucial for social interactions. Here we investigated whether processing self-face and self-body images can be explained by the brain's prediction of sensory events, based on regularities in the given context. We measured evoked cortical responses while participants observed alternating sequences of self-face or other-face images (experiment 1) and self-body or other-body images (experiment 2), which were embedded in an identity-irrelevant task. In experiment 1, the expected sequences were violated by deviant morphed images, which contained 33%, 66% or 100% of the self-face when the other's face was expected (and vice versa). In experiment 2, the anticipated sequences were violated by deviant images of the self when the other's image was expected (and vice versa), or by two deviant images composed of pictures of the self-face attached to the other's body, or the other's face attached to the self-body. This manipulation allowed control of the prediction error associated with the self or the other's image. Deviant self-images (but not deviant images of the other) elicited a visual mismatch response (vMMR)--a cortical index of violations of regularity. This was source localised to face and body related visual, sensorimotor and limbic areas and had amplitude proportional to the amount of deviance from the self-image. We provide novel evidence that self-processing can be described by the brain's prediction error system, which accounts for self-bias in visual processing. These findings are discussed in the light of recent predictive coding models of self-processing.

A frontal attention mechanism in the visual mismatch negativity

•

We examine frontal mechanisms underlying the visual mismatch negativity.

•

EEG and fMRI activity was examined in respect to unattended oddball stimuli.

•

Left inferior frontal gyrus was associated with changes in the stimuli.

•

Our findings correspond to similarly implicated regions in the auditory domain.

Automatic detection of environmental change is a core component of attention. The mismatch negativity (MMN), an electrophysiological marker of this mechanism, has been studied prominently in the auditory domain, with cortical generators identified in temporal and frontal regions. Here, we combined electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) to assess whether the underlying frontal regions associated with auditory change detection also play a role in visual change detection. Twenty healthy young adults completed a visual MMN task in separate EEG and fMRI sessions. Region of interest analyses were conducted on left and right middle frontal (MFG) and inferior frontal (IFG) gyri, i.e., the frontal areas identified as potential auditory MMN generators. A significant increase in activation was observed in the left IFG and MFG in response to blocks containing deviant stimuli. These findings suggest that a frontal mechanism is involved in the detection of change in the visual MMN. Our results support the notion that frontal mechanisms underlie attention switching, as measured via MMN, across multiple modalities.

Emotional context facilitates cortical prediction error responses

In the predictive coding framework, mismatch negativity (MMN) is regarded a correlate of the prediction error that occurs when top-down predictions conflict with bottom-up sensory inputs. Expression-related MMN is a relatively novel construct thought to reflect a prediction error specific to emotional processing, which, however, has not yet been tested directly. Our paradigm includes both neutral and emotional deviants, thereby allowing for investigating whether expression-related MMN is emotion-specific or unspecifically arises from violations of a given sequence. Twenty healthy participants completed a visual sequence oddball task where they were presented with (1) sequence deviants, (2) emotional sequence deviants, and (3) emotional deviants. Mismatch components were assessed at ventral occipitotemporal scalp sites and analyzed regarding their amplitudes, spatiotemporal profiles, and neuronal sources. Expression-related MMN could be clearly separated from its neutral counterpart in all investigated aspects. Specifically, expression-related MMN showed enhanced amplitude, shorter latency, and different neuronal sources. Our results, therefore, provide converging evidence for a quantitative specificity of expression-related MMN and seems to provide an opportunity to study prediction error during preattentive emotional processing. Our neurophysiological evidence ultimately suggests that a basic cognitive operator, the prediction error, is enhanced at the cortical level by processing of emotionally salient stimuli.

Automatic prediction regarding the next state of a visual object: Electrophysiological indicators of prediction match and mismatch

Behavioral phenomena such as representational momentum suggest that the brain can automatically predict the next state of a visual object, based on sequential rules embedded in its preceding spatiotemporal context. To identify electrophysiological indicators of automatic visual prediction in terms of prediction match and mismatch, we recorded event-related brain potentials (ERPs) while participants passively viewed three types of task-irrelevant sequences of a bar stimulus: (1) an oddball sequence, which contained a sequential rule defined by stimulus repetition, providing repetition-rule-conforming (standard) and -violating (deviant) stimuli; (2) a rotating-oddball sequence, which contained a sequential rule defined by stimulus change (i.e., rotation), providing change-rule-conforming (regular) and -violating (irregular) stimuli; and (3) a random sequence, which did not contain a sequential rule, providing a neutral (control) stimulus. This protocol allowed us to expect that (1) an ERP effect that reflects a prediction-mismatch process should be exclusively observed in both the deviant-minus-control and irregular-minus-control comparisons and (2) an ERP effect that reflects a prediction-match process should be exclusively observed in both the standard-minus-control and regular-minus-control comparisons. The results showed that the ERP effect that met the criterion for prediction mismatch was an occipito-temporal negative deflection at around 170-300ms (visual mismatch negativity), while the ERP effect that met the criterion for prediction match was a frontal/central negative deflection at around 150-270ms (probably, the reduction of P2). These two contrasting ERP effects support a hypothetical view that automatic visual prediction would involve both an increase in the neural response to prediction-incongruent (i.e., novel) events and a decrease in the neural response to prediction-congruent (i.e., redundant) events. This article is part of a Special Issue entitled SI: Prediction and Attention.

Visual mismatch negativity: a predictive coding view

An increasing number of studies investigate the visual mismatch negativity (vMMN) or use the vMMN as a tool to probe various aspects of human cognition. This paper reviews the theoretical underpinnings of vMMN in the light of methodological considerations and provides recommendations for measuring and interpreting the vMMN. The following key issues are discussed from the experimentalist's point of view in a predictive coding framework: (1) experimental protocols and procedures to control "refractoriness" effects; (2) methods to control attention; (3) vMMN and veridical perception.

Functional fractionation of the stimulus-driven attention network

A novel, salient event in the environment powerfully captures attention. This stimulus-driven attentional capture not only includes orienting of attention toward the event, but also an evaluative process to determine the behavioral significance and appropriate response to the event. Whereas a network of human brain regions composed of prefrontal and temporoparietal regions have been associated with stimulus-driven attention, the neural substrates of orienting have never been teased apart from those of evaluative processes. Here we used fMRI to measure the human brain's response to the temporally extended presentations of salient, task-irrelevant stimuli, and found a clear functional dissociation in the stimulus-driven attention network; the anterior insula and cingulate cortex showed transient orienting responses to the onsets and offsets of the stimuli, whereas the temporoparietal cortex exhibited sustained activity throughout event evaluation. The lateral prefrontal cortex was implicated in both attentional and evaluative processes, pointing to its central, integrative role in stimulus-driven attention.

Oblique effect in visual mismatch negativity

We investigated whether visual orientation anisotropies (known as oblique effect) exist in non-attended visual changes using event-related potentials (ERP). We recorded visual mismatch negativity (vMMN) which signals violation of sequential regularities. In the visual periphery unattended, task-irrelevant Gábor patches were displayed in an oddball sequence while subjects performed a tracking task in the central field. A moderate change (50°) in the orientation of stimuli revealed no consistent change-related components. However, we found orientation-related differences around 170 ms in occipito-temporal areas in the amplitude of the ERPs evoked by standard stimuli. In a supplementary experiment we determined the amount of orientation difference that is needed for change detection in an active, attended paradigm. Results exhibited the classical oblique effect; subjects detected 10° deviations from cardinal directions, while threshold from oblique directions was 17°. These results provide evidence that perception of change could be accomplished at significantly smaller thresholds, than what elicits vMMN. In Experiment 2 we increased the orientation change to 90°. Deviant-minus-standard difference was negative in occipito-parietal areas, between 120 and 200 ms after stimulus onset. VMMNs to changes from cardinal angles were larger and more sustained than vMMNs evoked by changes from oblique angles. Changes from cardinal orientations represent a more detectable signal for the automatic change detection system than changes from oblique angles, thus increased vMMN to these “larger” deviances might be considered a variant of the magnitude of deviance effect rarely observed in vMMN studies.

Visual mismatch negativity in the dorsal stream is independent of concurrent visual task difficulty

The manipulation of attention can produce mismatch negativity-like components that are not necessarily connected to the unintentional sensory registration of the violation of probability-based regularity. For clinical purposes, attentional bias should be quantified because it can vary substantially among subjects and can decrease the specificity of the examination. This experiment targets the role of attention in the generation of visual mismatch negativity (vMMN). The visual regularity was generated by a sequence of two radial motions while subjects focused on visual tasks in the central part of the display. Attentional load was systematically varied and had three levels, no-load, easy, and difficult. Rare, deviant, and frequent standard motions were presented with a 10/60 ratio in oddball sequences. Data from 12 subjects was recorded from 64 channels and processed. vMMN was identified within the interval of 142-198 ms. The mean amplitude was evaluated during the aforementioned interval in the parietal and fronto-central regions. A general linear model for repeated measures was applied to the mean amplitude with a three-factor design and showed a significant difference [F (1, 11) = 17.40, p = 0.002] between standard and deviant stimuli and between regions [F (1, 11) = 8.40, p = 0.01]; however, no significant effect of the task [F (2, 22) = 1.26, p = 0.30] was observed. The unintentional detection of irregularity during the processing of the visual motion was independent of the attentional load associated with handling the central visual task. The experiment did not demonstrate an effect of attentional load manipulation on mismatch negativity (MMN) induced by the motion-sequence, which supports the clinical utility of this examination. However, used stimulation paradigm should be further optimized to generate mismatch negativity that is stable enough to be usable not only for group comparisons but also for a single subject assessment.

Altered visual information processing systems in bipolar disorder: evidence from visual MMN and P3

Objective: Mismatch negativity (MMN) and P3 are unique ERP components that provide objective indices of human cognitive functions such as short-term memory and prediction. Bipolar disorder (BD) is an endogenous psychiatric disorder characterized by extreme shifts in mood, energy, and ability to function socially. BD patients usually show cognitive dysfunction, and the goal of this study was to access their altered visual information processing via visual MMN (vMMN) and P3 using windmill pattern stimuli.

Methods: Twenty patients with BD and 20 healthy controls matched for age, gender, and handedness participated in this study. Subjects were seated in front of a monitor and listened to a story via earphones. Two types of windmill patterns (standard and deviant) and white circle (target) stimuli were randomly presented on the monitor. All stimuli were presented in random order at 200-ms durations with an 800-ms inter-stimulus interval. Stimuli were presented at 80% (standard), 10% (deviant), and 10% (target) probabilities. The participants were instructed to attend to the story and press a button as soon as possible when the target stimuli were presented. Event-related potentials (ERPs) were recorded throughout the experiment using 128-channel EEG equipment. vMMN was obtained by subtracting standard from deviant stimuli responses, and P3 was evoked from the target stimulus.

Results: Mean reaction times for target stimuli in the BD group were significantly higher than those in the control group. Additionally, mean vMMN-amplitudes and peak P3-amplitudes were significantly lower in the BD group than in controls.

Conclusions: Abnormal vMMN and P3 in patients indicate a deficit of visual information processing in BD, which is consistent with their increased reaction time to visual target stimuli.

Significance: Both bottom-up and top-down visual information processing are likely altered in BD.

Object-related regularities are processed automatically: evidence from the visual mismatch negativity

One of the most challenging tasks of our visual systems is to structure and integrate the enormous amount of incoming information into distinct coherent objects. It is an ongoing debate whether or not the formation of visual objects requires attention. Implicit behavioral measures suggest that object formation can occur for task-irrelevant and unattended visual stimuli. The present study investigated pre-attentive visual object formation by combining implicit behavioral measures and an electrophysiological indicator of pre-attentive visual irregularity detection, the visual mismatch negativity (vMMN) of the event-related potential. Our displays consisted of two symmetrically arranged, task-irrelevant ellipses, the objects. In addition, there were two discs of either high or low luminance presented on the objects, which served as targets. Participants had to indicate whether the targets were of the same or different luminance. In separate conditions, the targets either usually were enclosed in the same object or in two different objects (standards). Occasionally, the regular target-to-object assignment was changed (deviants). That is, standards and deviants were exclusively defined on the basis of the task-irrelevant target-to-object assignment but not on the basis of some feature regularity. Although participants did not notice the regularity nor the occurrence of the deviation in the sequences, task-irrelevant deviations resulted in increased reaction times. Moreover, compared with physically identical standard displays deviating target-to-object assignments elicited a negative potential in the 246–280 ms time window over posterio-temporal electrode positions which was identified as vMMN. With variable resolution electromagnetic tomography (VARETA) object-related vMMN was localized to the inferior temporal gyrus. Our results support the notion that the visual system automatically structures even task-irrelevant aspects of the incoming information into objects.

Event-related potentials elicited by pre-attentive emotional changes in temporal context

The ability to detect emotional change in the environment is essential for adaptive behavior. The current study investigated whether event-related potentials (ERPs) can reflect emotional change in a visual sequence. To assess pre-attentive processing, we examined visual mismatch negativity (vMMN): the negative potentials elicited by a deviant (infrequent) stimulus embedded in a sequence of standard (frequent) stimuli. Participants in two experiments pre-attentively viewed visual sequences of Japanese kanji with different emotional connotations while ERPs were recorded. The visual sequence in Experiment 1 consisted of neutral standards and two types of emotional deviants with a strong and weak intensity. Although the results indicated that strongly emotional deviants elicited more occipital negativity than neutral standards, it was unclear whether these negativities were derived from emotional deviation in the sequence or from the emotional significance of the deviants themselves. In Experiment 2, the two identical emotional deviants were presented against different emotional standards. One type of deviants was emotionally incongruent with the standard and the other type of deviants was emotionally congruent with the standard. The results indicated that occipital negativities elicited by deviants resulted from perceptual changes in a visual sequence at a latency of 100–200 ms and from emotional changes at latencies of 200–260 ms. Contrary to the results of the ERP experiment, reaction times to deviants showed no effect of emotional context; negative stimuli were consistently detected more rapidly than were positive stimuli. Taken together, the results suggest that brain signals can reflect emotional change in a temporal context.

Brain correlates of automatic visual change detection

A number of studies support the presence of visual automatic detection of change, but little is known about the brain generators involved in such processing and about the modulation of brain activity according to the salience of the stimulus. The study presented here was designed to locate the brain activity elicited by unattended visual deviant and novel stimuli using fMRI. Seventeen adult participants were presented with a passive visual oddball sequence while performing a concurrent visual task. Variations in BOLD signal were observed in the modality-specific sensory cortex, but also in non-specific areas involved in preattentional processing of changing events. A degree-of-deviance effect was observed, since novel stimuli elicited more activity in the sensory occipital regions and at the medial frontal site than small changes. These findings could be compared to those obtained in the auditory modality and might suggest a "general" change detection process operating in several sensory modalities.

fMRI investigation of visual change detection in adults with autism

People with autism spectrum disorders (ASD) may show unusual reactions to unexpected changes that appear in their environment. Although several studies have highlighted atypical auditory change processing in ASD, little is known in this disorder about the brain processes involved in visual automatic change detection. The present fMRI study was designed to localize brain activity elicited by unexpected visual changing stimuli in adults with ASD compared to controls. Twelve patients with ASD and 17 healthy adults participated in the experiment in which subjects were presented with a visual oddball sequence while performing a concurrent target detection task. Combined results across participants highlight the involvement of both occipital (BA 18/19) and frontal (BA 6/8) regions during visual change detection. However, adults with ASD display greater activity in the bilateral occipital cortex and in the anterior cingulate cortex (ACC) associated with smaller activation in the superior and middle frontal gyri than controls. A psychophysiological interaction (PPI) analysis was performed with ACC as the seed region and revealed greater functionally connectivity to sensory regions in ASD than in controls, but less connectivity to prefrontal and orbito-frontal cortices. Thus, compared to controls, larger sensory activation associated with reduced frontal activation was seen in ASD during automatic visual change detection. Atypical psychophysiological interactions between frontal and occipital regions were also found, congruent with the idea of atypical connectivity between these regions in ASD. The atypical involvement of the ACC in visual change detection can be related to abnormalities previously observed in the auditory modality, thus supporting the hypothesis of an altered general mechanism of change detection in patients with ASD that would underlie their unusual reaction to change.

Gender differences in preattentive processing of facial expressions: an ERP study

To investigate gender differences in pre-attentive processing of facial expressions we recorded the expression mismatch negativity (EMMN) in the deviant-standard-reverse oddball paradigm. For female participants, sad faces elicited larger EMMN than happy faces, but this difference disappeared in the left hemisphere. For male participants, EMMN was not modulated by facial expressions, regardless of in the left or right hemispheres. While the source analysis indicated that for both genders prefrontal activations were observed for sad EMMN, more sources were involved for male than female participants. For happy EMMN, the current sources were located in the occipital lobe and parietal lobe for females and the temporal lobe and the frontal lobe for males. The present findings emphasized the importance of considering gender as a factor in the study of the pre-attentive processing of facial expressions.

Atypical visual change processing in children with autism: an electrophysiological study

Children with Autism Spectrum Disorder (ASD) may display atypical behaviors in reaction to unattended changes that occur in all sensory modalities. Atypical automatic auditory change processing has been highlighted in ASD via the analysis of mismatch negativity (MMN). The present study investigated visual deviancy detection in children with ASD in order to determine whether unusual reactions to change operate in other sensory modalities. Twelve children with ASD were presented with a passive visual oddball paradigm using dynamic stimuli. Compared to controls, children with ASD showed an earlier visual mismatch response, suggesting a hypersensitivity to visual deviancy. This study is thus consistent with the hypothesis of the existence of "general" atypical change detection processing in children with ASD that might contribute to their intolerance of change.

Impact of lower- vs. upper-hemifield presentation on automatic colour-deviance detection: a visual mismatch negativity study

The automatic processing of deviances from the temporal context of the visual environment has become an important topic in visual cognitive sciences, which is often investigated using the visual mismatch negativity (vMMN). This event-related potential (ERP) component is elicited by an irregular stimulus (e.g., a red disc) presented in a series of stimuli (e.g., green discs) comprising a temporal regularity (e.g., colour repetition). We determined the influence of lower- vs. upper-hemifield presentation of the irregular stimulus on the vMMN while using whole-field stimulus displays controlling for sustained shifts in spatial attention. Deviances presented in the lower hemifield elicited a larger vMMN than the ones presented in the upper hemifield at a latency of 200-280ms. However, this asymmetry was preceded by deviance-related hemifield effects already emerging at an earlier latency (110-150ms), where upper-hemifield deviances elicited a positive potential but lower-hemifield deviances did not. With variable resolution electromagnetic tomography (VARETA) early deviance-related activity was localised to retinotopically organised regions of the visual cortex (BA 17/18) and vMMN-sources were localised to the middle/superior occipital gyrus, to higher areas along the temporal visual stream, but also to BA 17/18. We argue that the upper/lower-hemifield vMMN asymmetry relies at least partially on the hemifield-dependent differential sensitivity of early deviance-related activity generated in retinotopically organised regions of the visual cortex. However, a superior automatic processing of deviances presented in the lower visual hemifield may also contribute to the effect.

Automatic prediction error responses to hands with unexpected laterality: an electrophysiological study

Little is known about how the human brain keeps track of body parts in the visual field. Here we show that unattended images of right/left hands elicit a mismatch response when they violate a regularity established by repeated visual presentations of the other hand. In a visual oddball experiment we found mismatch responses to hands with unexpected laterality (e.g. left versus predicted right hand) in the periphery of the visual field. Unexpected left hands were processed predominantly in the contralateral superior parietal cortex, whereas unexpected right hands evoked differential activity in the contralateral superior parietal, ventral premotor, prefrontal and temporal areas, indicating a more elaborate automatic processing of the dominant hand. The amplitude of the differential activity to the right hand correlated with handedness test scores. Our results reveal the continuous monitoring of the left or right identity of hands, which is prerequisite to the ability to automatically transform observed actions into the observer's ego-centric spatial reference frame.

Electrophysiological correlates of automatic visual change detection in school-age children

Automatic stimulus-change detection is usually investigated in the auditory modality by studying Mismatch Negativity (MMN). Although the change-detection process occurs in all sensory modalities, little is known about visual deviance detection, particularly regarding the development of this brain function throughout childhood. The aim of the present study was to examine the maturation of the electrophysiological response to unattended deviant visual stimuli in 11-year-old children. Twelve children and 12 adults were presented with a passive visual oddball paradigm using dynamic stimuli involving changes in form and motion. Visual Mismatch responses were identified over occipito-parietal sites in both groups but they displayed several differences. In adults the response clearly culminated at around 210 ms whereas in children three successive negative deflections were evidenced between 150 and 330 ms. Moreover, the main mismatch response in children was characterized by a positive component peaking over occipito-parieto-temporal regions around 450 ms after deviant stimulus onset. The findings showed that the organization of the vMMN response is not mature in 11-year-old children and that a longer time is still necessary to process simple visual deviancy at this late stage of child development.

Unintentional temporal context-based prediction of emotional faces: an electrophysiological study

The ability to extract sequential regularities embedded in the temporal context or temporal structure of sensory events and to predict upcoming events based on the extracted sequential regularities plays a central role in human cognition. In the present study, we demonstrate that, without any intention, upcoming emotional faces can be predicted based on sequential regularities, by showing that prediction error responses as reflected by visual mismatch negativity (MMN), an event-related brain potential (ERP) component, were evoked in response to emotional faces that violated a regular alternation pattern of 2 emotional faces (fearful and happy faces) under a situation where the emotional faces themselves were unrelated to the participant's task. Face-inversion and negative-bias effects in the visual MMN further indicated the involvement of holistic face representations. In addition, through successive source analyses of the visual MMN, it was revealed that the prediction error responses were composed of activations mainly in the face-responsible visual extrastriate areas and the prefrontal areas. The present results provide primary evidence for the existence of the unintentional temporal context-based prediction of biologically relevant visual stimuli as well as empirical support for the major engagement of the visual and prefrontal areas in unintentional temporal context-based prediction in vision.

Human Visual System Automatically Encodes Sequential Regularities of Discrete Events

For our adaptive behavior in a dynamically changing environment, an essential task of the brain is to automatically encode sequential regularities inherent in the environment into a memory representation. Recent studies in neuroscience have suggested that sequential regularities embedded in discrete sensory events are automatically encoded into a memory representation at the level of the sensory system. This notion is largely supported by evidence from investigations using auditory mismatch negativity (auditory MMN), an event-related brain potential (ERP) correlate of an automatic memory-mismatch process in the auditory sensory system. However, it is still largely unclear whether or not this notion can be generalized to other sensory modalities. The purpose of the present study was to investigate the contribution of the visual sensory system to the automatic encoding of sequential regularities using visual mismatch negativity (visual MMN), an ERP correlate of an automatic memory-mismatch process in the visual sensory system. To this end, we conducted a sequential analysis of visual MMN in an oddball sequence consisting of infrequent deviant and frequent standard stimuli, and tested whether the underlying memory representation of visual MMN generation contains only a sensory memory trace of standard stimuli (trace-mismatch hypothesis) or whether it also contains sequential regularities extracted from the repetitive standard sequence (regularity-violation hypothesis). The results showed that visual MMN was elicited by first deviant (deviant stimuli following at least one standard stimulus), second deviant (deviant stimuli immediately following first deviant), and first standard (standard stimuli immediately following first deviant), but not by second standard (standard stimuli immediately following first standard). These results are consistent with the regularity-violation hypothesis, suggesting that the visual sensory system automatically encodes sequential regularities. In combination with a wide range of auditory MMN studies, the present study highlights the critical role of sensory systems in automatically encoding sequential regularities when modeling the world.

Visual Object Representations Can Be Formed outside the Focus of Voluntary Attention: Evidence from Event-related Brain Potentials

There is an ongoing debate whether visual object representations can be formed outside the focus of voluntary attention. Recently, implicit behavioral measures suggested that grouping processes can occur for task-irrelevant visual stimuli, thus supporting theories of preattentive object formation (e.g., Lamy, D., Segal, H., & Ruderman, L. Grouping does not require attention. Perception and Psychophysics, 68, 17–31, 2006; Russell, C., & Driver, J. New indirect measures of “inattentive” visual grouping in a change-detection task. Perception and Psychophysics, 67, 606–623, 2005). We developed an ERP paradigm that allows testing for visual grouping when neither the objects nor its constituents are related to the participant's task. Our paradigm is based on the visual mismatch negativity ERP component, which is elicited by stimuli deviating from a regular stimulus sequence even when the stimuli are ignored. Our stimuli consisted of four pairs of colored discs that served as objects. These objects were presented isochronously while participants were engaged in a task related to the continuously presented fixation cross. Occasionally, two color deviances occurred simultaneously either within the same object or across two different objects. We found significant ERP differences for same- versus different-object deviances, supporting the notion that forming visual object representations by grouping can occur outside the focus of voluntary attention. Also our behavioral experiment, in which participants responded to color deviances—thus, this time the discs but, again, not the objects were task relevant—showed that the object status matters. Our results stress the importance of early grouping processes for structuring the perceptual world.

Cortical dynamics of the visual change detection process

In this study, the cortical dynamics of the visual change detection process were investigated using an oddball paradigm similar to that used in auditory mismatch negativity studies. When subjects watched a silent movie, color stimuli were presented using 280 dual color LEDs arranged along the frame of the video screen. Task-irrelevant red and blue color stimuli were presented randomly at a probability of 10% and 90%, respectively, in one session and vice versa for the other one, and we traced brain responses using magnetoencephalography. Results show that activation in the middle occipital gyrus (MOG) was significantly enhanced for the infrequent stimulus, while early activities in Brodmann's area 17/18 were comparable for the frequent and infrequent stimuli. These results suggest that automatic visual change detection is associated with the MOG activity.

Responses in a manual tracking and visual detection task

To investigate interference between the sustained and occasional attention required in driving, performance of 36 participants engaged in a focal manual tracking task and a peripheral detection task, representing sustained and occasional attention, respectively, was studied. Error ratio, tracking distance, tracking speed, and root mean squared tracking distance error were taken for manual tracking, and response times were measured on the detection task. Analysis indicated that multiple tasks or highly sustained attentional demands preceded lower performance on sustained attention tasks. Performance on occasional attention tasks after multiple tasks may also decrease, but occasional attention performance improved when the participants were engaged concurrently in a task involving higher sustained attention. Furthermore, the association between sustained and occasional attention was strengthened as the number of tasks or the demands of sustained attention increased.

Automatic change detection during the performance of a continuous visual task

The brain's automatic detection of change within the visual environment can be examined by recording visual mismatch negativity (vMMN), a negative deflection in the visual event-related potential evoked by infrequent deviant stimuli presented within a sequence of common standard stimuli. In this study, we examine whether automatic visual change detection occurs even when the visual cortex is concurrently processing other visual information at the focus of attention. We ensured that attention was strongly engaged upon a highly demanding and continuous central task, while at the same time presenting in the peripheral field a sequence of task-irrelevant standard stimuli interspersed by infrequent deviant stimuli. A significant vMMN was evoked. However, decreasing the rarity of deviation resulted in the abolition of the vMMN response.

The neural basis of agency: an fMRI study

Agency, a feeling that the self is the cause of action, has a strong relationship to the processing of discrepancies between the predicted multi-sensory feedback from one's intended action and its actual outcome (hereafter, agency error). Although previous studies have explored the neural basis of agency by assessing the brain's response to agency error, the effects found are confounded by two types of error irrelevant to agency: a mismatch between different sensory inputs in general (sensory mismatch, SM error) and a basic response to any type of prediction error (oddball error). In this functional magnetic resonance imaging study, we identified the neural response specific to agency error by dissociating it from responses to SM and oddball errors. Subjects played a game in which they controlled an on-screen character. Neural responses to rare events of violated control and congruency between types of audio-visual feedback were compared to dissociate agency from SM error. In a separate session, subjects viewed repetitive motions of the character, and neural responses to rare events of unpredictable change in movement were identified as related to oddball error. Agency-error-specific activation was observed in the supplementary motor area (SMA), left cerebellum, right posterior parietal cortex (PPC), and right extrastriate body area (EBA). Oddball errors also activated areas near the PPC and EBA peaks. SM errors activated the pre-SMA and the right posterior superior temporal sulcus. Our results suggest that the SMA, cerebellum, and some parts of the PPC and EBA serve as the neural bases of agency.

Atypical modulation of cognitive control by arousal in autism

We examined the effects of viewing high-arousal pictures on regional brain activations elicited by a cognitive control task in participants with high-functioning autism and neurotypical controls. Specifically, using event-related functional magnetic resonance imaging, we assessed the effects of brief presentations of highly arousing pictures (i.e., both very pleasant and very unpleasant) on the processing of stimuli requiring cognitive control. Similar to previous findings, when stimuli with high cognitive control demands were preceded by low-arousal pictures, individuals with autism demonstrated regional brain activations that were comparable to neurotypical control individuals. When the presentation of the cognitive control stimuli was preceded by high-arousal pictures, however, the control group was characterized by relatively greater activation in the right lateral midfrontal cortex in response to cognitive control stimuli. In contrast, preceding high-arousal stimuli did not modulate activity elicited in this region by cognitive control stimuli in the autism group. Differential modulation of right lateral midfrontal activation by high-arousal stimuli in autism is consistent with the "inefficiency model" of brain functioning in autism spectrum disorders, and contributes to a growing body of evidence that autism may be characterized by anomalous sensitivity of cognitive control brain regions to social-emotional context.

Functional-MRI correlates of cued slow-eye-closure and task non-responsiveness during visuomotor tracking

Slow-eye-closure and task non-responsiveness are important behavioural markers of microsleeps. This paper presents preliminary results on the neural correlates of voluntary slow-eye-closure and voluntary non-responsiveness during performance of a continuous visuomotor tracking task. Functional-MRI (fMRI), EEG, eye video, and tracking responses were recorded from 5 normal subjects while they performed a continuous visuomotor tracking task inside an MRI scanner for 10 min. During this time, they were cued to simultaneously stop tracking and slowly close their eyes or stop tracking without eye-closure several times. Analysis of fMRI data revealed several regions involved in cued slow-eye-closure and cued task non-responsiveness, including occipito-parietal visual regions, midline default mode regions, and fronto-parietal attention regions. These results will be of considerable value in the interpretation of changes in BOLD activation and EEG activity associated with behavioural microsleeps.