Brain correlates of automatic visual change detection

Emotional visual mismatch negativity: a joint investigation of social and non-social dimensions in adults with autism

Unusual behaviors and brain activity to socio-emotional stimuli have been reported in Autism Spectrum Disorder (ASD). Atypical reactivity to change and intolerance of uncertainty are also present, but little is known on their possible impact on facial expression processing in autism. The visual mismatch negativity (vMMN) is an electrophysiological response automatically elicited by changing events such as deviant emotional faces presented among regular neutral faces. While vMMN has been found altered in ASD in response to low-level changes in simple stimuli, no study has investigated this response to visual social stimuli. Here two deviant expressions were presented, neutral and angry, embedded in a sequence of repetitive neutral stimuli. vMMN peak analyses were performed for latency and amplitude in early and late time windows. The ASD group presented smaller amplitude of the late vMMN to both neutral and emotional deviants compared to the typically developed adults (TD) group, and only the TD group presented a sustained activity related to emotional change (i.e., angry deviant). Source reconstruction of the vMMNs further revealed that any change processing elicited a reduced activity in ASD group compared to TD in the saliency network, while the specific processing emotional change elicited activity in the temporal region and in the insula. This study confirms atypical change processing in ASD and points to a specific difficulty in the processing of emotional changes, potentially playing a crucial role in social interaction deficits. Nevertheless, these results require to be further replicated with a greater sample size and generalized to other emotional expressions.

Visual mismatch negativity to masked stimuli presented at very brief presentation rates

Mismatch negativity (MMN) has been characterised as a 'pre-attentive' component of an event-related potential (ERP) that is related to discrimination and error prediction processes. The aim of the current experiment was to establish whether visual MMN could be recorded to briefly presented, backward and forward masked visual stimuli, given both below and above levels of subjective experience. Evidence of visual MMN elicitation in the absence of the ability to consciously report stimuli would provide strong evidence for the automaticity of the visual MMN mechanism. Using an oddball paradigm, two stimuli that differed in orientation from each other, a + and an ×, were presented on a computer screen. Electroencephalogram (EEG) was recorded from nine participants (six females), mean age 21.4 years. Results showed that for stimuli that were effectively masked at 7 ms presentation, there was little variation in the ERPs evoked to standard and deviant stimuli or in the subtraction waveform employed to delineate the visual MMN. At 14 ms stimulus presentation, when participants were able to report stimulus presence, an enhanced negativity at around 175 and 305 ms was observed to the deviant and was evident in the subtraction waveform. However, some of the difference observed in the ERPs can be attributed to stimulus characteristics, as the use of a 'lonely' deviant protocol revealed attenuated visual MMN components at 14 ms stimulus presentation. Overall, results suggest that some degree of conscious attention is required before visual MMN components emerge, suggesting visual MMN is not an entirely pre-attentive process.

Facial Expression Related vMMN: Disentangling Emotional from Neutral Change Detection

Detection of changes in facial emotional expressions is crucial to communicate and to rapidly and automatically process possible threats in the environment. Recent studies suggest that expression-related visual mismatch negativity (vMMN) reflects automatic processing of emotional changes. In the present study we used a controlled paradigm to investigate the specificity of emotional change-detection. In order to disentangle specific responses to emotional deviants from that of neutral deviants, we presented neutral expression as standard stimulus (p = 0.80) and both angry and neutral expressions as deviants (p = 0.10, each). In addition to an oddball sequence, an equiprobable sequence was presented, to control for refractoriness and low-level differences. Our results showed that in an early time window (100–200 ms), the controlled vMMN was greater than the oddball vMMN only for the angry deviant, suggesting the importance of controlling for refractoriness and stimulus physical features in emotion related studies. Within the controlled vMMN, angry and neutral deviants both elicited early and late peaks occurring at 140 and 310 ms, respectively, but only the emotional vMMN presented sustained amplitude after each peak. By directly comparing responses to emotional and neutral deviants, our study provides evidence of specific activity reflecting the automatic detection of emotional change. This differs from broader “visual” change processing, and suggests the involvement of two partially-distinct pre-attentional systems in the detection of changes in facial expressions.

Dynamic emotion perception and prior expectancy

Social interactions require the ability to rapidly perceive emotion from various incoming dynamic, multisensory cues. Prior expectations reduce incoming emotional information and direct attention to cues that are aligned with what is expected. Studies to date have investigated the prior expectancy effect using static emotional images, despite the fact that dynamic stimuli would represent greater ecological validity. The objective of the study was to create a novel functional magnetic resonance imaging (fMRI) paradigm to examine the influence of prior expectations on naturalistic emotion perception. For this purpose, we developed a dynamic emotion perception task, which consisted of audio-visual videos that carry emotional information congruent or incongruent with prior expectations. The results show that emotional congruency was associated with activity in prefrontal regions, amygdala, and putamen, whereas emotional incongruency was associated with activity in temporoparietal junction and mid-cingulate gyrus. Supported by the behavioural results, our findings suggest that prior expectations are reinforced after repeated experience and learning, whereas unexpected emotions may rely on fast change detection processes. The results from the current study are compatible with the notion that the ability to automatically detect unexpected changes in complex dynamic environments allows for adaptive behaviours in potentially advantageous or threatening situations.

Brain basis of communicative actions in language

Although language is a key tool for communication in social interaction, most studies in the neuroscience of language have focused on language structures such as words and sentences. Here, the neural correlates of speech acts, that is, the actions performed by using language, were investigated with functional magnetic resonance imaging (fMRI). Participants were shown videos, in which the same critical utterances were used in different communicative contexts, to Name objects, or to Request them from communication partners. Understanding of critical utterances as Requests was accompanied by activation in bilateral premotor, left inferior frontal and temporo-parietal cortical areas known to support action-related and social interactive knowledge. Naming, however, activated the left angular gyrus implicated in linking information about word forms and related reference objects mentioned in critical utterances. These findings show that understanding of utterances as different communicative actions is reflected in distinct brain activation patterns, and thus suggest different neural substrates for different speech act types.

Automatic detection of trustworthiness of the face: a visual mismatch negativity study

Recognizing intentions of strangers from facial cues is crucial in everyday social interactions. Recent studies demonstrated enhanced event-related potential (ERP) responses to untrustworthy compared to trustworthy faces. The aim of the present study was to investigate the electrophysiological correlates of automatic processing of trustworthiness cues in a visual oddball paradigm in two consecutive experimental blocks. In one block, frequent trustworthy (p = 0.9) and rare untrustworthy face stimuli (p = 0.1) were briefly presented on a computer screen with each stimulus consisting of four peripherally positioned faces. In the other block stimuli were presented with reversed probabilities enabling the comparison of ERPs evoked by physically identical deviant and standard stimuli. To avoid attentional effects participants engaged in a central detection task. Analyses of deviant minus standard difference waveforms revealed that deviant untrustworthy but not trustworthy faces elicited the visual mismatch negativity (vMMN) component. The present results indicate that adaptation occurred to repeated unattended trustworthy (but not untrustworthy) faces, i.e., an automatic expectation was elicited towards trustworthiness signals, which was violated by deviant untrustworthy faces. As an evolutionary adaptive mechanism, the observed fast detection of trustworthiness-related social facial cues may serve as the basis of conscious recognition of reliable partners.

The inferior colliculus is involved in deviant sound detection as revealed by BOLD fMRI

Rapid detection of deviant sounds is a crucial property of the auditory system because it increases the saliency of biologically important, unexpected sounds. The oddball paradigm in which a deviant sound is randomly interspersed among a train of standard sounds has been traditionally used to study this property in mammals. Currently, most human studies have only revealed the involvement of cortical regions in this property. Recently, several animal electrophysiological studies have reported that neurons in the inferior colliculus (IC) exhibit reduced responses to a standard sound but restore their responses at the occurrence of a deviant sound (i.e., stimulus-specific adaptation or SSA), suggesting that the IC may also be involved in deviance detection. However, by adopting an invasive method, these animal studies examined only a limited number of neurons. Although SSA appears to be more prominent in the external cortical nuclei of the IC for frequency deviant, a thorough investigation of this property throughout the IC using other deviants and efficient imaging techniques may provide more comprehensive information on this important phenomenon. In this study, blood-oxygen-level-dependent (BOLD) fMRI with a large field of view was applied to investigate the role of the IC in deviance detection. Two sound tokens that had identical frequency spectrum but temporally inverted profiles were used as the deviant and standard. A control experiment showed that these two sounds evoked the same responses in the IC when they were separately presented. Two oddball experiments showed that the deviant induced higher responses than the standard (by 0.41±0.09% and 0.41±0.10%, respectively). The most activated voxels were in the medial side of the IC in both oddball experiments. The results clearly demonstrated that the IC is involved in deviance detection. BOLD fMRI detection of increased activities in the medial side of the IC to the deviant revealed the highly adaptive nature of a substantial population of neurons in this region, probably those that belong to the rostral or dorsal cortex of the IC. These findings highlighted the complexity of auditory information processing in the IC and may guide future studies of the functional organizations of this subcortical structure.

Oblique effect in visual mismatch negativity

We investigated whether visual orientation anisotropies (known as oblique effect) exist in non-attended visual changes using event-related potentials (ERP). We recorded visual mismatch negativity (vMMN) which signals violation of sequential regularities. In the visual periphery unattended, task-irrelevant Gábor patches were displayed in an oddball sequence while subjects performed a tracking task in the central field. A moderate change (50°) in the orientation of stimuli revealed no consistent change-related components. However, we found orientation-related differences around 170 ms in occipito-temporal areas in the amplitude of the ERPs evoked by standard stimuli. In a supplementary experiment we determined the amount of orientation difference that is needed for change detection in an active, attended paradigm. Results exhibited the classical oblique effect; subjects detected 10° deviations from cardinal directions, while threshold from oblique directions was 17°. These results provide evidence that perception of change could be accomplished at significantly smaller thresholds, than what elicits vMMN. In Experiment 2 we increased the orientation change to 90°. Deviant-minus-standard difference was negative in occipito-parietal areas, between 120 and 200 ms after stimulus onset. VMMNs to changes from cardinal angles were larger and more sustained than vMMNs evoked by changes from oblique angles. Changes from cardinal orientations represent a more detectable signal for the automatic change detection system than changes from oblique angles, thus increased vMMN to these “larger” deviances might be considered a variant of the magnitude of deviance effect rarely observed in vMMN studies.

More attention when speaking: does it help or does it hurt?

Paying selective attention to a word in a multi-word utterance results in a decreased probability of error on that word (benefit), but an increased probability of error on the other words (cost). We ask whether excitation of the prefrontal cortex helps or hurts this cost. One hypothesis (the resource hypothesis) predicts a decrease in the cost due to the deployment of more attentional resources, while another (the focus hypothesis) predicts even greater costs due to further fine-tuning of selective attention. Our results are more consistent with the focus hypothesis: prefrontal stimulation caused a reliable increase in the benefit and a marginal increase in the cost of selective attention. To ensure that the effects are due to changes to the prefrontal cortex, we provide two checks: We show that the pattern of results is quite different if, instead, the primary motor cortex is stimulated. We also show that the stimulation-related benefits in the verbal task correlate with the stimulation-related benefits in an N-back task, which is known to tap into a prefrontal function. Our results shed light on how selective attention affects language production, and more generally, on how selective attention affects production of a sequence over time.

Altered visual information processing systems in bipolar disorder: evidence from visual MMN and P3

Objective: Mismatch negativity (MMN) and P3 are unique ERP components that provide objective indices of human cognitive functions such as short-term memory and prediction. Bipolar disorder (BD) is an endogenous psychiatric disorder characterized by extreme shifts in mood, energy, and ability to function socially. BD patients usually show cognitive dysfunction, and the goal of this study was to access their altered visual information processing via visual MMN (vMMN) and P3 using windmill pattern stimuli.

Methods: Twenty patients with BD and 20 healthy controls matched for age, gender, and handedness participated in this study. Subjects were seated in front of a monitor and listened to a story via earphones. Two types of windmill patterns (standard and deviant) and white circle (target) stimuli were randomly presented on the monitor. All stimuli were presented in random order at 200-ms durations with an 800-ms inter-stimulus interval. Stimuli were presented at 80% (standard), 10% (deviant), and 10% (target) probabilities. The participants were instructed to attend to the story and press a button as soon as possible when the target stimuli were presented. Event-related potentials (ERPs) were recorded throughout the experiment using 128-channel EEG equipment. vMMN was obtained by subtracting standard from deviant stimuli responses, and P3 was evoked from the target stimulus.

Results: Mean reaction times for target stimuli in the BD group were significantly higher than those in the control group. Additionally, mean vMMN-amplitudes and peak P3-amplitudes were significantly lower in the BD group than in controls.

Conclusions: Abnormal vMMN and P3 in patients indicate a deficit of visual information processing in BD, which is consistent with their increased reaction time to visual target stimuli.

Significance: Both bottom-up and top-down visual information processing are likely altered in BD.