MMN in the visual modality: a review

Automatic processing of unattended mean emotion: Evidence from visual mismatch responses

The mean emotion from multiple facial expressions can be extracted rapidly and precisely. However, it remains debated whether mean emotion processing is automatic which can occur under no attention. To address this question, we used a passive oddball paradigm and recorded event-related brain potentials when participants discriminated the changes in the central fixation while a set of four faces was presented in the periphery. The face set consisted of one happy and three angry expressions (mean negative) or one angry and three happy expressions (mean positive), and the mean negative and mean positive face sets were shown with a probability of 20% (deviant) and 80% (standard) respectively in the sequence, or the vice versa. The cluster-based permutation analyses showed that the visual mismatch negativity started early at around 92 ms and was also observed in later time windows when the mean emotion was negative, while a mismatch positivity was observed at around 168-266 ms when the mean emotion was positive. The results suggest that there might be different mechanisms underlying the processing of mean negative and mean positive emotions. More importantly, the brain can detect the changes in the mean emotion automatically, and ensemble coding for multiple facial expressions can occur in an automatic fashion without attention.

The developmental cognitive mechanism of learning algebraic rules from the dual-process theory perspective

Rule learning is an important ability that enables human beings to adapt to nature and develop civilizations. There have been many discussions on the mechanism and characteristics of algebraic rule learning, but there are still controversies due to the lack of theoretical guidance. Based on the dual-process theory, this study discussed the following arguments for algebraic rule learning across human and animal studies: whether algebraic rule learning is simply Type 1 processing, whether algebraic rule learning is a domain-general ability, whether algebraic rule learning is shared by humans and animals, and whether an algebraic rule is learned consciously. Moreover, we propose that algebraic rule learning is possibly a cognitive process that combines both Type 1 and Type 2 processing. Further exploration is required to establish the essence and neural basis of algebraic rule learning.

Spontaneously emerging internal models of visual sequences combine abstract and event-specific information in the prefrontal cortex

When exposed to sensory sequences, do macaque monkeys spontaneously form abstract internal models that generalize to novel experiences? Here, we show that neuronal populations in macaque ventrolateral prefrontal cortex jointly encode visual sequences by separate codes for the specific pictures presented and for their abstract sequential structure. We recorded prefrontal neurons while macaque monkeys passively viewed visual sequences and sequence mismatches in the local-global paradigm. Even without any overt task or response requirements, prefrontal populations spontaneously form representations of sequence structure, serial order, and image identity within distinct but superimposed neuronal subspaces. Representations of sequence structure rapidly update following single exposure to a mismatch sequence, while distinct populations represent mismatches for sequences of different complexity. Finally, those representations generalize across sequences following the same repetition structure but comprising different images. These results suggest that prefrontal populations spontaneously encode rich internal models of visual sequences reflecting both content-specific and abstract information.

Enhanced electrophysiological responses to explicitly predicted and pre-imagined inputs, with confirmation from online decoding with neuro-feedback

Neural responses to sensory inputs can scale with the likelihood of encountering the input. This is consistent with the predictive coding framework, in that the human brain is expected to be less responsive to predicted inputs. Typically, however, prediction is not explicitly measured. It is inferred from the probability of encountering an event. When an input is explicitly predicted, responses to predicted inputs can be enhanced. Here, we ask if this effect can be ascribed to a generic priming effect, from pre-cogitating about one of two possible inputs. Consistent with this, we find that P300s (a relatively late event-related potential measured with electroencephalography) are greater for explicitly predicted audio and visual inputs, and that this effect cannot be distinguished from a priming effect from pre-imagining audio or visual presentations. Evidence indicates that participants engaged in pre-imagining presentations, as we were able to decode online what type of presentation (audio or visual) they were imagining with a high success rate (approx. 73%), and we encouraged compliance with neuro-feedback regarding this success rate. Our data confirm that human cortex can be more responsive to inputs that have been subject to pre-cogitation-including explicit predictions. This highlights that while anticipatory processes can reduce responding to likely inputs, they can also enhance responding to explicitly predicted inputs.

Effect of motion state variability on error-related potentials during continuous feedback paradigms and their consequences for classification

BACKGROUND

An erroneous motion would elicit the error-related potential (ErrP) when humans monitor the behavior of the external devices. This EEG modality has been largely applied to brain-computer interface in an active or passive manner with discrete visual feedback. However, the effect of variable motion state on ErrP morphology and classification performance raises concerns when the interaction is conducted with continuous visual feedback.

NEW METHOD

In the present study, we designed a cursor control experiment. Participants monitored a continuously moving cursor to reach the target on one side of the screen. Motion state varied multiple times with two factors: (1) motion direction and (2) motion speed. The effects of these two factors on the morphological characteristics and classification performance of ErrP were analyzed. Furthermore, an offline simulation was performed to evaluate the effectiveness of the proposed extended ErrP-decoder in resolving the interference by motion direction changes.

RESULTS

The statistical analyses revealed that motion direction and motion speed significantly influenced the amplitude of feedback-ERN and frontal-Pe components, while only motion direction significantly affected the classification performance.

COMPARISON WITH EXISTING METHODS

Significant deviation was found in ErrP detection utilizing classical correct-versus-erroneous event training. However, this bias can be alleviated by 16% by the extended ErrP-decoder.

CONCLUSION

The morphology and classification performance of ErrP signal can be affected by motion state variability during continuous feedback paradigms. The results enhance the comprehension of ErrP morphological components and shed light on the detection of BCI's error behavior in practical continuous control.

A multimodal cortical network of sensory expectation violation revealed by fMRI

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

Spectral and phase-coherence correlates of impaired auditory mismatch negativity (MMN) in schizophrenia: A MEG study

BACKGROUND

Reduced auditory mismatch negativity (MMN) is robustly impaired in schizophrenia. However, mechanisms underlying dysfunctional MMN generation remain incompletely understood. This study aimed to examine the role of evoked spectral power and phase-coherence towards deviance detection and its impairments in schizophrenia.

METHODS

Magnetoencephalography data was collected in 16 male schizophrenia patients and 16 male control participants during an auditory MMN paradigm. Analyses of event-related fields (ERF), spectral power and inter-trial phase-coherence (ITPC) focused on Heschl's gyrus, superior temporal gyrus, inferior/medial frontal gyrus and thalamus.

RESULTS

MMNm ERF amplitudes were reduced in patients in temporal, frontal and subcortical regions, accompanied by decreased theta-band responses, as well as by a diminished gamma-band response in auditory cortex. At theta/alpha frequencies, ITPC to deviant tones was reduced in patients in frontal cortex and thalamus. Patients were also characterized by aberrant responses to standard tones as indexed by reduced theta-/alpha-band power and ITPC in temporal and frontal regions. Moreover, stimulus-specific adaptation was decreased at theta/alpha frequencies in left temporal regions, which correlated with reduced MMNm spectral power and ERF amplitude. Finally, phase-reset of alpha-oscillations after deviant tones in left thalamus was impaired, which correlated with impaired MMNm generation in auditory cortex. Importantly, both non-rhythmic and rhythmic components of spectral activity contributed to the MMNm response.

CONCLUSIONS

Our data indicate that deficits in theta-/alpha- and gamma-band activity in cortical and subcortical regions as well as impaired spectral responses to standard sounds could constitute potential mechanisms for dysfunctional MMN generation in schizophrenia, providing a novel perspective towards MMN deficits in the disorder.

Short-term neuronal and synaptic plasticity act in synergy for deviance detection in spiking networks

Sensory areas of cortex respond more strongly to infrequent stimuli when these violate previously established regularities, a phenomenon known as deviance detection (DD). Previous modeling work has mainly attempted to explain DD on the basis of synaptic plasticity. However, a large fraction of cortical neurons also exhibit firing rate adaptation, an underexplored potential mechanism. Here, we investigate DD in a spiking neuronal network model with two types of short-term plasticity, fast synaptic short-term depression (STD) and slower threshold adaptation (TA). We probe the model with an oddball stimulation paradigm and assess DD by evaluating the network responses. We find that TA is sufficient to elicit DD. It achieves this by habituating neurons near the stimulation site that respond earliest to the frequently presented standard stimulus (local fatigue), which diminishes the response and promotes the recovery (global fatigue) of the wider network. Further, we find a synergy effect between STD and TA, where they interact with each other to achieve greater DD than the sum of their individual effects. We show that this synergy is caused by the local fatigue added by STD, which inhibits the global response to the frequently presented stimulus, allowing greater recovery of TA-mediated global fatigue and making the network more responsive to the deviant stimulus. Finally, we show that the magnitude of DD strongly depends on the timescale of stimulation. We conclude that highly predictable information can be encoded in strong local fatigue, which allows greater global recovery and subsequent heightened sensitivity for DD.

Attentional impairment in Parkinson's disease is modulated by side of onset: Neurophysiological evidence

OBJECTIVE

Neurophysiological studies exploring involuntary attention have reported that electroencephalographic (EEG) measures can indicate impaired neural processing from initial stages of Parkinson's disease (PD). Since involuntary attention is regulated by right hemisphere networks and PD generally initiates its motor symptomatology unilaterally, whether involuntary attention is impaired depending on the onset side of PD remains unknown.

METHODS

We compared the neurophysiological correlates of involuntary attention among a PD group with left-side onset (L-PD), a PD group with right-side onset (R-PD) symptomatology, and a healthy control group (HC). All participants performed an auditory involuntary attention task while a digital EEG was recorded.

RESULTS

Our main finding was a reduction both in the P3a amplitude and evoked delta-theta phase alignment in the L-PD group compared to the HC. Further, there was a significant correlation between P3a amplitude and disease duration in the R-PD, but not in the L-PD group. Behaviorally, both clinical groups, and in particular L-PD, showed reduced orientation towards novel stimuli, and no reduction of distraction effects during the experiment.

CONCLUSIONS

Our results indicate that involuntary attention is differentially impaired in patients with left side onset of symptoms. Involuntary attention impairment might be present from initial stages of left onset PD and become progressively compromised in patients with right onset PD.

SIGNIFICANCE

The onset side of symptomatology should be considered for attentional impairment in PD.

Intelligence and Visual Mismatch Negativity: Is Pre-Attentive Visual Discrimination Related to General Cognitive Ability?

EEG has been used for decades to identify neurocognitive processes related to intelligence. Evidence is accumulating for associations with neural markers of higher-order cognitive processes (e.g., working memory); however, whether associations are specific to complex processes or also relate to earlier processing stages remains unclear. Addressing these issues has implications for improving our understanding of intelligence and its neural correlates. The MMN is an ERP that is elicited when, within a series of frequent standard stimuli, rare deviant stimuli are presented. As stimuli are typically presented outside the focus of attention, the MMN is suggested to capture automatic pre-attentive discrimination processes. However, the MMN and its relation to intelligence has largely only been studied in the auditory domain, thus preventing conclusions about the involvement of automatic discrimination processes in humans' dominant sensory modality-vision. EEG was recorded from 50 healthy participants during a passive visual oddball task that presented simple sequence violations and deviations within a more complex hidden pattern. Signed area amplitudes and fractional area latencies of the visual MMN were calculated with and without Laplacian transformation. Correlations between visual MMN and intelligence (Raven's Advanced Progressive Matrices) were of negligible to small effect sizes, differed critically between measurement approaches, and Bayes Factors provided anecdotal to substantial evidence for the absence of an association. We discuss differences between the auditory and visual MMN, the implications of different measurement approaches, and offer recommendations for further research in this evolving field.

Sensory Target Detection at Local and Global Timescales Reveals a Hierarchy of Supramodal Dynamics in the Human Cortex

To ensure survival in a dynamic environment, the human neocortex monitors input streams from different sensory organs for important sensory events. Which principles govern whether different senses share common or modality-specific brain networks for sensory target detection? We examined whether complex targets evoke sustained supramodal activity while simple targets rely on modality-specific networks with short-lived supramodal contributions. In a series of hierarchical multisensory target detection studies (n = 77, of either sex) using EEG, we applied a temporal cross-decoding approach to dissociate supramodal and modality-specific cortical dynamics elicited by rule-based global and feature-based local sensory deviations within and between the visual, somatosensory, and auditory modality. Our data show that each sense implements a cortical hierarchy orchestrating supramodal target detection responses, which operate at local and global timescales in successive processing stages. Across different sensory modalities, simple feature-based sensory deviations presented in temporal vicinity to a monotonous input stream triggered a mismatch negativity-like local signal which decayed quickly and early, whereas complex rule-based targets tracked across time evoked a P3b-like global neural response which generalized across a late time window. Converging results from temporal cross-modality decoding analyses across different datasets, we reveal that global neural responses are sustained in a supramodal higher-order network, whereas local neural responses canonically thought to rely on modality-specific regions evolve into short-lived supramodal activity. Together, our findings demonstrate that cortical organization largely follows a gradient in which short-lived modality-specific as well as supramodal processes dominate local responses, whereas higher-order processes encode temporally extended abstract supramodal information fed forward from modality-specific cortices.SIGNIFICANCE STATEMENT Each sense supports a cortical hierarchy of processes tracking deviant sensory events at multiple timescales. Conflicting evidence produced a lively debate around which of these processes are supramodal. Here, we manipulated the temporal complexity of auditory, tactile, and visual targets to determine whether cortical local and global ERP responses to sensory targets share cortical dynamics between the senses. Using temporal cross-decoding, we found that temporally complex targets elicit a supramodal sustained response. Conversely, local responses to temporally confined targets typically considered modality-specific rely on early short-lived supramodal activation. Our finding provides evidence for a supramodal gradient supporting sensory target detection in the cortex, with implications for multiple fields in which these responses are studied (e.g., predictive coding, consciousness, and attention).

Functional brain connectivity during exposure to the scale and color of interior built environments

Understanding brain activity linked to built environment exposure is important, as it may affect underlying cognitive, perceptual, and emotional processes, which have a critical influence in our daily life. As our time spent inside buildings is rising, and mental health problems have become more prevalent, it is important we investigate how design characteristics of the built environment impact brain function. In this study, we utilized electroencephalography to understand whether the design elements of scale and color of interior built environments modulate functional brain connectivity (i.e., brain network communication). Using a Cave Automatic Virtual Environment, while controlling indoor environmental quality responsible for physiological comfort, healthy adult participants aged 18-55 years (66 for scale, subset of 18 for color), were exposed to context-neutral indoor room scenes presented for two-minutes each. Our results show that both enlarging and reducing scale enhanced theta connectivity across the left temporoparietal region and right frontal region. We also found when reducing the built environment scale, there was a network exhibiting greater high-gamma connectivity, over the right frontoparietal region. For color, the condition (blue) contrasted to our achromatic control (white) increased theta connectivity in the frontal hemispheres. These findings identify a link between theta and gamma oscillations during exposure to the scale and color of the built environment, showing that design characteristics of the built environment could affect our cognitive processes and mental health. This suggests that, through the design of buildings, we may be able to mediate performance and health outcomes, which could lead to major health and economic benefits for society.

Research on Effective Recognition of Alarm Signals in Human-Machine System Based on Cognitive Neural Experiments

The reasonable design of the alarm signal in the man-machine system is one of the important factors that determine the occurrence of safety accidents. Neuroergonomics provides a new perspective for the study of the cognitive process of alarm signals, which can reveal the mechanism of human perception of visual alarm signals from the cognitive level of the brain, thereby identifying the effectiveness of alarm signals. The article's research simulated the human-machine system for heat dissipation of new energy vehicles, used the automatic control interface of the cooling water system as the stimulus material, and used the event-related potential technology in cognitive neuroscience for experimental verification. The experimental results showed that: three kinds of alarm signals (color, color + shape, color + orientation) all induce visual mismatch waves, and the effective response of human to the alarm signal is color + orientation, color + shape, color from small to large, which provides a reference for the design of the alarm signal of the man-machine system.

The impact of musical training in symbolic and non-symbolic audiovisual judgements of magnitude

Quantity estimation can be represented in either an analog or symbolic manner and recent evidence now suggests that analog and symbolic representation of quantities interact. Nonetheless, those two representational forms of quantities may be enhanced by convergent multisensory information. Here, we elucidate those interactions using high-density electroencephalography (EEG) and an audiovisual oddball paradigm. Participants were presented simultaneous audiovisual tokens in which the co-varying pitch of tones was combined with the embedded cardinality of dot patterns. Incongruencies were elicited independently from symbolic and non-symbolic modality within the audio-visual percept, violating the newly acquired rule that “the higher the pitch of the tone, the larger the cardinality of the figure.” The effect of neural plasticity in symbolic and non-symbolic numerical representations of quantities was investigated through a cross-sectional design, comparing musicians to musically naïve controls. Individual’s cortical activity was reconstructed and statistically modeled for a predefined time-window of the evoked response (130–170 ms). To summarize, we show that symbolic and non-symbolic processing of magnitudes is re-organized in cortical space, with professional musicians showing altered activity in motor and temporal areas. Thus, we argue that the symbolic representation of quantities is altered through musical training.

Caught in the ACTS: Defining Abstract Cognitive Task Sequences as an Independent Process

Cognitive neuroscience currently conflates the study of serial responses (e.g., delay match to sample/nonsample, n-back) with the study of sequential operations. In this essay, our goal is to define and disentangle the latter, termed abstract cognitive task sequences (ACTS). Existing literatures address tasks requiring serial events, including procedural learning of implicit motor responses, statistical learning of predictive relationships, and judgments of attributes. These findings do not describe the behavior and underlying mechanism required to succeed at remembering to evaluate color, then shape; or to multiply, then add. A new literature is needed to characterize these sorts of second-order cognitive demands of studying a sequence of operations. Our second goal is to characterize gaps in knowledge related to ACTS that merit further investigation. In the following sections, we define more precisely what we mean by ACTS and suggest research questions' further investigation would be positioned to address.

Neural prediction errors depend on how an expectation was formed

When a visual event is unexpected, because it violates a train of repeated events, it excites a greater positive electrical potential at sensors positioned above occipital-parietal human brain regions (the P300). Such events can also seem to have an increased duration relative to repeated (implicitly expected) events. However, recent behavioural evidence suggests that when events are unexpected because they violate a declared prediction-a guess-there is an opposite impact on duration perception. The neural consequences of incorrect declared predictions have not been examined. We replicated the finding whereby repetition violating events elicit a larger P300 response. However, we found that events that violated a declared prediction entrained an opposite pattern of response-a smaller P300. These data suggest that the neural consequences of a violated prediction are not uniform but depend on how the prediction was formed.

Event-related potentials evidence for long-term audiovisual representations of phonemes in adults

The presence of long-term auditory representations for phonemes has been well-established. However, since speech perception is typically audiovisual, we hypothesized that long-term phoneme representations may also contain information on speakers' mouth shape during articulation. We used an audiovisual oddball paradigm in which, on each trial, participants saw a face and heard one of two vowels. One vowel occurred frequently (standard), while another occurred rarely (deviant). In one condition (neutral), the face had a closed, non-articulating mouth. In the other condition (audiovisual violation), the mouth shape matched the frequent vowel. Although in both conditions stimuli were audiovisual, we hypothesized that identical auditory changes would be perceived differently by participants. Namely, in the neutral condition, deviants violated only the audiovisual pattern specific to each block. By contrast, in the audiovisual violation condition, deviants additionally violated long-term representations for how a speaker's mouth looks during articulation. We compared the amplitude of mismatch negativity (MMN) and P3 components elicited by deviants in the two conditions. The MMN extended posteriorly over temporal and occipital sites even though deviants contained no visual changes, suggesting that deviants were perceived as interruptions in audiovisual, rather than auditory only, sequences. As predicted, deviants elicited larger MMN and P3 in the audiovisual violation compared to the neutral condition. The results suggest that long-term representations of phonemes are indeed audiovisual.

Fronto-occipital mismatch responses in pre-attentive detection of visual changes: Implication on a generic brain network underlying Mismatch Negativity (MMN)

Current theories of pre-attentive change detection suggest a regularity or prediction violation mechanism involving a frontotemporal network. Modulations of the early inferior frontal cortex (IFC) mismatch response representing the effort in comparing a stimulus to the prediction, the superior temporal cortex (STC) response indicating deviance detection, and the late IFC response representing prediction model updating were consistently demonstrated in auditory change detection using event-related optical signal (EROS). If the prediction violation hypothesis is universal, a generic neural mechanism should be found in all sensory modalities. We postulated a generic fronto-sensory cortical network underlying the prediction violation mechanism: the IFC is responsible for non-modality-specific prediction processes while the sensory cortices are responsible for modality-specific error signal generation process. This study examined the involvement of the IFC-occipital cortex (OC) network in visual pre-attentive change detection. The EROS mismatch responses to deviant bar arrays violating a fixed orientation regularity (low in regularity abstractness) were compared to that of deviant violating a rotational orientation regularity (high in abstractness) while the information available for establishing the prediction model was manipulated by varying the number of standards preceding the deviants. Modulations of the IFCOC mismatch response patterns by abstractness and train length reflected the processing demands on the prediction processes and were similar to that of the IFC-STC network in auditory change detection. These findings demonstrated that the fronto-sensory cortical network is not unique to auditory pre-attentive change detection and provided supports for a universal neural mechanism across sensory modalities as suggested by the prediction violation hypothesis.

EEG-based visual deviance detection in freely behaving mice

The mouse is widely used as an experimental model to study visual processing. To probe how the visual system detects changes in the environment, functional paradigms in freely behaving mice are strongly needed. We developed and validated the first EEG-based method to investigate visual deviance detection in freely behaving mice. Mice with EEG implants were exposed to a visual deviant detection paradigm that involved changes in light intensity as standard and deviant stimuli. By subtracting the standard from the deviant evoked waveform, deviant detection was evident as bi-phasic negativity (starting around 70 ms) in the difference waveform. Additionally, deviance-associated evoked (beta/gamma) and induced (gamma) oscillatory responses were found. We showed that the results were stimulus-independent by applying a "flip-flop" design and the results showed good repeatability in an independent measurement. Together, we put forward a validated, easy-to-use paradigm to measure visual deviance processing in freely behaving mice.

EEG and behavioral correlates of attentional processing while walking and navigating naturalistic environments

The capacity to regulate one's attention in accordance with fluctuating task demands and environmental contexts is an essential feature of adaptive behavior. Although the electrophysiological correlates of attentional processing have been extensively studied in the laboratory, relatively little is known about the way they unfold under more variable, ecologically-valid conditions. Accordingly, this study employed a 'real-world' EEG design to investigate how attentional processing varies under increasing cognitive, motor, and environmental demands. Forty-four participants were exposed to an auditory oddball task while (1) sitting in a quiet room inside the lab, (2) walking around a sports field, and (3) wayfinding across a university campus. In each condition, participants were instructed to either count or ignore oddball stimuli. While behavioral performance was similar across the lab and field conditions, oddball count accuracy was significantly reduced in the campus condition. Moreover, event-related potential components (mismatch negativity and P3) elicited in both 'real-world' settings differed significantly from those obtained under laboratory conditions. These findings demonstrate the impact of environmental factors on attentional processing during simultaneously-performed motor and cognitive tasks, highlighting the value of incorporating dynamic and unpredictable contexts within naturalistic designs.

Auditory, Visual, and Cross-Modal Mismatch Negativities in the Rat Auditory and Visual Cortices

When the brain tries to acquire an elaborate model of the world, multisensory integration should contribute to building predictions based on the various pieces of information, and deviance detection should repeatedly update these predictions by detecting “errors” from the actual sensory inputs. Accumulating evidence such as a hierarchical organization of the deviance-detection system indicates that the deviance-detection system can be interpreted in the predictive coding framework. Herein, we targeted mismatch negativity (MMN) as a type of prediction-error signal and investigated the relationship between multisensory integration and MMN. In particular, we studied whether and how cross-modal information processing affected MMN in rodents. We designed a new surface microelectrode array and simultaneously recorded visual and auditory evoked potentials from the visual and auditory cortices of rats under anesthesia. Then, we mapped MMNs for five types of deviant stimuli: single-modal deviants in (i) the visual oddball and (ii) auditory oddball paradigms, eliciting single-modal MMN; (iii) congruent audio-visual deviants, (iv) incongruent visual deviants, and (v) incongruent auditory deviants in the audio-visual oddball paradigm, eliciting cross-modal MMN. First, we demonstrated that visual MMN exhibited deviance detection properties and that the first-generation focus of visual MMN was localized in the visual cortex, as previously reported in human studies. Second, a comparison of MMN amplitudes revealed a non-linear relationship between single-modal and cross-modal MMNs. Moreover, congruent audio-visual MMN exhibited characteristics of both visual and auditory MMNs—its latency was similar to that of auditory MMN, whereas local blockage of N-methyl-D-aspartic acid receptors in the visual cortex diminished it as well as visual MMN. These results indicate that cross-modal information processing affects MMN without involving strong top-down effects, such as those of prior knowledge and attention. The present study is the first electrophysiological evidence of cross-modal MMN in animal models, and future studies on the neural mechanisms combining multisensory integration and deviance detection are expected to provide electrophysiological evidence to confirm the links between MMN and predictive coding theory.

Perceptual and cognitive processes in augmented reality - comparison between binocular and monocular presentations

In the present study, we investigated the difference between monocular augmented reality (AR) and binocular AR in terms of perception and cognition by using a task that combines the flanker task with the oddball task. A right- or left-facing arrowhead was presented as a central stimulus at the central vision, and participants were instructed to press a key only when the direction in which the arrowhead faced was a target. In a small number of trials, arrowheads that were facing in the same or opposite direction (flanker stimuli) were presented beside the central stimulus binocularly or monocularly as an AR image. In the binocular condition, the flanker stimuli were presented to both eyes, and, in the monocular condition, only to the dominant eye. The results revealed that participants could respond faster in the binocular condition than in the monocular one; however, only when the flanker stimuli were in the opposite direction was the response faster in the monocular condition. Moreover, the results of event-related brain potentials (ERPs) showed that all stimuli were processed in both the monocular and the binocular conditions in the perceptual stage; however, the influence of the flanker stimuli was attenuated in the monocular condition in the cognitive stage. The influence of flanker stimuli might be more unstable in the monocular condition than in the binocular condition, but more precise examination should be conducted in a future study.

Psilocybin Induces Aberrant Prediction Error Processing of Tactile Mismatch Responses-A Simultaneous EEG-FMRI Study

As source of sensory information, the body provides a sense of agency and self/non-self-discrimination. The integration of bodily states and sensory inputs with prior beliefs has been linked to the generation of bodily self-consciousness. The ability to detect surprising tactile stimuli is essential for the survival of an organism and for the formation of mental body representations. Despite the relevance for a variety of psychiatric disorders characterized by altered body and self-perception, the neurobiology of these processes is poorly understood. We therefore investigated the effect of psilocybin (Psi), known to induce alterations in self-experience, on tactile mismatch responses by combining pharmacological manipulations with simultaneous electroencephalography-functional magnetic resonance imaging (EEG-fMRI) recording. Psi reduced activity in response to tactile surprising stimuli in frontal regions, the visual cortex, and the cerebellum. Furthermore, Psi reduced tactile mismatch negativity EEG responses at frontal electrodes, associated with alterations of body- and self-experience. This study provides first evidence that Psi alters the integration of tactile sensory inputs through aberrant prediction error processing and highlights the importance of the 5-HT2A system in tactile deviancy processing as well as in the integration of bodily and self-related stimuli. These findings may have important implications for the treatment of psychiatric disorders characterized by aberrant bodily self-awareness.

Do categorical representations modulate early automatic visual processing? A visual mismatch-negativity study

Perceptual categorization is an important cognitive function. In the auditory domain, categorization already occurs within the first 200 ms of information processing, as indexed by the mismatch negativity. Here, we assessed the characteristics of the visual mismatch negativity (vMMN) elicited during the categorization of previously unknown visual stimuli. To examine this, we used five-dot patterns with characteristics that allow for the formation of categories through rotation and reflection but not through other physical properties. To assess whether or not between-category and within-category vMMN differ in amplitude, the data was analyzed with the Bayesian approach. We observed that both between-category and within-category deviants elicited a vMMN, but that both vMMNs were comparable in magnitude. This finding suggests that abstract categorical representations are not always automatically processed at early visual stages and demonstrates limitations of generalization from the auditory domain to visual domain.

Deficits in Visual Processing During Hypoxia as Evidenced by Visual Mismatch Negativity

INTRODUCTION: Hypoxia is an ever-present threat in tactical aviation and gained recent attention due to its putative role in physiological episodes. Previous work has demonstrated that hypoxia negatively impacts a variety of sensory, cognitive, and motor systems. In particular, the visual system is one of the earliest systems affected by hypoxia. While the majority of previous studies have relied on self-report and behavioral testing, the use of event-related potentials as a novel tool to monitor responses to low oxygen in humans has recently been investigated. Specifically, ERP components that are evoked passively in response to unattended changes in background sensory stimulation have been explored.METHOD: Subjects (N 28) completed a continuous visuomotor tracking task while EEG was recorded. During the tracking task, a series of standard color checkerboard patterns were presented in the periphery while occasionally a deviant color checkerboard was presented. The visual mismatch negativity (MMN) component was assessed in response to the deviant compared to the standard stimuli. Subjects completed two sessions in counterbalanced order that only differed by the oxygen concentration breathed (10.6% vs. 20.4%).RESULTS: Results demonstrated a significant reduction in the amplitude of the visual MMN under hypoxic compared to normoxic conditions, showing a 50% reduction in amplitude during hypoxia. Our results suggest that during low-oxygen exposure the ability to detect environmental changes and process sensory information is impaired.DISCUSSION: The visual MMN may represent an early and reliable predictor of sensory and cognitive deficits during hypoxia exposure, which may be of great use to the aviation community.Blacker KJ, Seech TR, Funke ME, Kinney MJ. Deficits in visual processing during hypoxia as evidenced by visual mismatch negativity. Aerosp Med Hum Perform. 2021; 92(5):326332.

Do categorical representations modulate early perceptual or later cognitive visual processing? An ERP study

Encoding of perceptual categorical information has been observed in later cognitive processing like memory encoding and maintenance, starting around 300 ms after stimulus onset (P300). However, it remains open whether categorical information is also encoded in early perceptual processing steps (reflected in the mismatch negativity component; vMMN). The main goal of this study was to assess the influence of categorical information on both early perceptual (i.e., vMMN component) and later cognitive (i.e., P300 component) processing within one paradigm. Hence, we combined an oddball paradigm with a delayed memory task. We used five-dot patterns belonging to different categories even though categorical information is not mirrored in their physical characteristics. Distinct vMMNs were observed for patterns belonging to the same as compared to different categories, suggesting that abstract categorical information was encoded during early perceptual processing. However, inconsistent with prior studies, we observed no effect of categories on the P300, indicating no additional encoding of categorical information in later cognitive stages of processing. Our findings emphasize that the encoding of categorical information depends on specific task demands and hence is more flexible and dynamic than previously suggested.

MEG reveals preference specific increases of sexual-image-evoked responses in paedophilic sexual offenders and healthy controls

OBJECTIVES

Paedophilic disorder is characterised by sexual attraction towards children. Classification of a counterpart as sexually attractive likely occurs rapidly, and involves both conscious and unconscious attentional and cognitive processes. Magnetoencephalography (MEG) is an imaging method especially well-suited to examine visual and attentional processes triggered by sexual images within the range of milliseconds.

METHODS

We investigated brain responses to sexual images depicting adults (frequent) and children (infrequent stimulus) in seventeen paedophilic patients with a history of child sexual offending (P + CSO) and twenty healthy controls (HC) during a passive visual oddball paradigm. Event-related fields (ERF) were measured to extract the magnetic visual mismatch negativity (vMMNm), and how it relates to the processing of different classes of sexual stimuli.

RESULTS

P + CSO exhibited significantly longer vMMNm latencies (100-180 ms post-stimulus) than HC. Moreover, P + CSO showed widespread increased amplitudes in response to child images starting from P3a and P3b components and lasting up to 400 ms post-stimulus presentation localised in frontal and temporal brain regions.

CONCLUSIONS

This study uncovers the first MEG differences in automatic change detection between P + CSO and HC during the presentation of subliminal sexual images of adults and children, contributing towards a better understanding of the neurobiological processes of P + CSO.

Spatially intermixed objects of different categories are parsed automatically

Our visual system is able to separate spatially intermixed objects into different categorical groups (e.g., berries and leaves) using the shape of feature distribution: Determining whether all objects belong to one or several categories depends on whether the distribution has one or several peaks. Despite the apparent ease of rapid categorization, it is a very computationally demanding task, given severely limited "bottlenecks" of attention and working memory capable of processing only a few objects at a time. Here, we tested whether this rapid categorical parsing is automatic or requires attention. We used the visual mismatch negativity (vMMN) ERP component known as a marker of automatic sensory discrimination. 20 volunteers (16 female, mean age-22.7) participated in our study. Loading participants' attention with a central task, we observed a substantial vMMN response to unattended background changes of categories defined by certain length-orientation conjunctions. Importantly, this occurred in conditions where the distributions of these features had several peaks and, hence, supported categorical separation. These results suggest that spatially intermixed objects are parsed into distinct categories automatically and give new insight into how the visual system can bypass the severe processing restrictions and form rich perceptual experience.

Selective Impairment of Processing Task-Irrelevant Emotional Faces in Cerebral Small Vessel Disease Patients

BACKGROUND

Few reports have implied electrophysiological alterations and neurocognitive abnormalities in patients with cerebral small vessel disease (CSVD), while no investigation is available regarding emotional processing. In the present study, pre-attentive processing of facial expressions was compared between CSVD sufferers and healthy controls using expression-related visual mismatch negativity (EMMN) as the indicator.

METHODS

A total of 22 CSVD patients (12 males) and 21 age-matched healthy controls (12 males) were recruited for neuropsychological and emotional assessments, as well as electroencephalogram recording and analysis. We employed an expression-related oddball paradigm to investigate automatic emotional processing, and a series of schematic emotional faces (neutral, happy, sad) unrelated to subject's task were present in the test to avoid low-level processing of facial features.

RESULTS

Although the distinctions of neuropsychological (MoCA and MMSE), emotional (GAD-7 and PHQ-9) and behavioral parameters (reaction time to target stimuli and response accuracy) did not reach significant levels, mean amplitudes of sad EMMN in time intervals of 150-250 ms and 250-350 ms were remarkably reduced in CSVD patients compared with healthy controls, but not for happy EMMN. Furthermore, in the control group, sad EMMN was demonstrated to be larger (more negative) than happy EMMN, while this interesting phenomenon disappeared in the CSVD group.

CONCLUSION

Our findings confirmed selective impairment of processing expressions which were task-irrelevant in CSVD patients, without the existence of negative bias (sad superiority) effect. The efficacy of EMMN as an electrophysiological evaluation marker of CSVD should be taken into account in future investigations.

A review on the electroencephalography markers of Stroop executive control processes

The present article on executive control addresses the issue of the locus of the Stroop effect by examining neurophysiological components marking conflict monitoring, interference suppression, and conflict resolution. Our goal was to provide an overview of a series of determining neurophysiological findings including neural source reconstruction data on distinct executive control processes and sub-processes involved in the Stroop task. Consistently, a fronto-central N2 component is found to reflect conflict monitoring processes, with its main neural generator being the anterior cingulate cortex (ACC). Then, for cognitive control tasks that involve a linguistic component like the Stroop task, the N2 is followed by a centro-posterior N400 and subsequently a late sustained potential (LSP). The N400 is mainly generated by the ACC and the prefrontal cortex (PFC) and is thought to reflect interference suppression, whereas the LSP plausibly reflects conflict resolution processes. The present overview shows that ERP constitute a reliable methodological tool for tracing with precision the time course of different executive processes and sub-processes involved in experimental tasks involving a cognitive conflict. Future research should shed light on the fine-grained mechanisms of control respectively involved in linguistic and non-linguistic tasks.

Detecting (Un)seen Change: The Neural Underpinnings of (Un)conscious Prediction Errors

Detecting changes in the environment is fundamental for our survival. According to predictive coding theory, detecting these irregularities relies both on incoming sensory information and our top-down prior expectations (or internal generative models) about the world. Prediction errors (PEs), detectable in event-related potentials (ERPs), occur when there is a mismatch between the sensory input and our internal model (i.e., a surprise event). Many changes occurring in our environment are irrelevant for survival and may remain unseen. Such changes, even if subtle, can nevertheless be detected by the brain without emerging into consciousness. What remains unclear is how these changes are processed in the brain at the network level. Here, we used a visual oddball paradigm in which participants engaged in a central letter task during electroencephalographic (EEG) recordings while presented with task-irrelevant high- or low-coherence background, random-dot motion. Critically, once in a while, the direction of the dots changed. After the EEG session, we confirmed that changes in motion direction at high- and low-coherence were visible and invisible, respectively, using psychophysical measurements. ERP analyses revealed that changes in motion direction elicited PE regardless of the visibility, but with distinct spatiotemporal patterns. To understand these responses, we applied dynamic causal modeling (DCM) to the EEG data. Bayesian Model Averaging showed visible PE relied on a release from adaptation (repetition suppression) within bilateral MT+, whereas invisible PE relied on adaptation at bilateral V1 (and left MT+). Furthermore, while feedforward upregulation was present for invisible PE, the visible change PE also included downregulation of feedback between right MT+ to V1. Our findings reveal a complex interplay of modulation in the generative network models underlying visible and invisible motion changes.

Disentangling top-down and bottom-up influences on blinks in the visual and auditory domain

Sensory input as well as cognitive factors can drive the modulation of blinking. Our aim was to dissociate sensory driven bottom-up from cognitive top-down influences on blinking behavior and compare these influences between the auditory and the visual domain. Using an oddball paradigm, we found a significant pre-stimulus decrease in blink probability for visual input compared to auditory input. Sensory input further led to an early post-stimulus blink increase in both modalities if a task demanded attention to the input. Only visual input caused a pronounced early increase without a task. In case of a target or the omission of a stimulus (as compared to standard input), an additional late increase in blink rate was found in the auditory and visual domain. This suggests that blink modulation must be based on the interpretation of the input, but does not need any sensory input at all to occur. Our results show a complex modulation of blinking based on top-down factors such as prediction and attention in addition to sensory-based influences. The magnitude of the modulation is mainly influenced by general attentional demands, while the latency of this modulation allows dissociating general from specific top-down influences that are independent of the sensory domain.

Fast Periodic Visual Stimulation indexes preserved semantic memory in healthy ageing

Behavioural studies investigating the preservation of semantic memory in healthy ageing have reported mixed findings. One suggested reason for this discrepancy is that the processes underpinning lexical access to semantic knowledge may be sensitive to ageing. It is therefore necessary to assess semantic memory utilising tasks that are not explicitly linguistic. In this study, a fast periodic visual stimulation (FPVS) paradigm coupled with EEG was used to assess the ability of younger and older adults to automatically distinguish between images by their semantic category. Participants were presented with a 6 Hz stream of images drawn from one semantic category except every fifth image (occurring at a rate of 1.2 Hz) which was drawn from an alternate semantic category. For both younger and older adults, results demonstrate successful and comparable semantic categorisation. This was detectable at the individual level for 71% and 72% of older and younger adults, respectively. Given the rapid presentation rate and absence of explicit instruction to categorise images, the task is unlikely to utilise linguistic strategies and suggests the maintenance of semantic memory in healthy ageing. Moreover, this study utilised mobile EEG equipment and short presentation times that would be suitable for practical application outside a research setting.

Attentional Interactions Between Vision and Hearing in Event-Related Responses to Crossmodal and Conjunct Oddballs

Are alternation and co-occurrence of stimuli of different sensory modalities conspicuous? In a novel audio-visual oddball paradigm, the P300 was used as an index of the allocation of attention to investigate stimulus- and task-related interactions between modalities. Specifically, we assessed effects of modality alternation and the salience of conjunct oddball stimuli that were defined by the co-occurrence of both modalities. We presented (a) crossmodal audio-visual oddball sequences, where both oddballs and standards were unimodal, but of a different modality (i.e., visual oddball with auditory standard, or vice versa), and (b) oddball sequences where standards were randomly of either modality while the oddballs were a combination of both modalities (conjunct stimuli). Subjects were instructed to attend to one of the modalities (whether part of a conjunct stimulus or not). In addition, we also tested specific attention to the conjunct stimuli. P300-like responses occurred even when the oddball was of the unattended modality. The pattern of event-related potential (ERP) responses obtained with the two crossmodal oddball sequences switched symmetrically between stimulus modalities when the task modality was switched. Conjunct oddballs elicited no oddball response if only one modality was attended. However, when conjunctness was specifically attended, an oddball response was obtained. Crossmodal oddballs capture sufficient attention even when not attended. Conjunct oddballs, however, are not sufficiently salient to attract attention when the task is unimodal. Even when specifically attended, the processing of conjunctness appears to involve additional steps that delay the oddball response.

Consciousness modulates the automatic change detection of masked emotional faces: Evidence from visual mismatch negativity

The ability to automatically detect changes in the environment is crucial for organisms to survive. In the visual system, changes in visual stimuli may evoke visual mismatch negativity (vMMN), an event-related potential (ERP) component that reflects automatic change detection. Previous studies that used visual masking to examine the effects of consciousness did not yield evidence that vMMN could be elicited by subliminal stimuli. However, these studies used relatively simple visual features. To further examine the role of consciousness in vMMN, the present study used emotional (happy and fearful) faces, which are biologically and socially significant visual stimuli. A passive oddball paradigm was employed, and we found that only fearful faces could evoke vMMN at a low consciousness level. Furthermore, the fear-related vMMN was enhanced by the consciousness level, and localized in regions of the brain associated with emotional face processing and the prefrontal cortex. We also found that the emotional visual mismatch oscillatory responses (vMORs) were associated with the enhancement of the alpha-band oscillation. Moreover, consciousness could weaken the happiness-related vMOR. These results suggested that changes to emotional faces-especially fearful faces-could be unconsciously detected by the brain. More importantly, this automatic change detection mechanism could be modulated by consciousness.

Prediction errors in surface segmentation are reflected in the visual mismatch negativity, independently of task and surface features

The visual system quickly registers perceptual regularities in the environment and responds to violations in these patterns. Errors of perceptual prediction are associated with electrocortical modulation, including the visual mismatch negativity (vMMN) and P2 event-related potential. One relatively unexplored question is whether these prediction error signals can encode higher-level properties such as surface segmentation, or whether they are limited to lower-level perceptual features. Using a roving standard paradigm, a triangle surface appeared either behind (featuring amodal contours) or in front of (featuring real contours) a second surface with hole-like windows. A surface layout appeared for two to five repetitions before switching to the other "deviant" layout; lighting and orientation of stimuli varied across presentations while remaining isoluminant. Observers responded when they detected a rare "pinched" triangle, which occasionally appeared. Cortical activity-reflected in mismatch responses affecting the P2-N2 and P300 amplitudes-was sensitive to a change in stimulus layout, when surfaces shifted position in depth, following several repetitions. Specifically, layout deviants led to a more negative P2-N2 complex at posterior electrodes, and greater P300 positivity at central sites. Independently of these signals of a deviant surface layout, further modulations of the P2 encoded differences between layouts and detection of the rare target stimulus. Comparison of the effect of preceding layout repetitions on this prediction error signal suggests that it is all or none and not graded with respect to the number of previous repetitions. We show that within the visual domain, unnoticed and task-irrelevant changes in visual surface segmentation leads to observable electrophysiological signals of prediction error that are dissociable from stimulus-specific encoding and lower-level perceptual processing.

Out of Focus: Facial Feedback Manipulation Modulates Automatic Processing of Unattended Emotional Faces

Facial expressions provide information about an individual's intentions and emotions and are thus an important medium for nonverbal communication. Theories of embodied cognition assume that facial mimicry and resulting facial feedback plays an important role in the perception of facial emotional expressions. Although behavioral and electrophysiological studies have confirmed the influence of facial feedback on the perception of facial emotional expressions, the influence of facial feedback on the automatic processing of such stimuli is largely unexplored. The automatic processing of unattended facial expressions can be investigated by visual expression-related MMN. The expression-related MMN reflects a differential ERP of automatic detection of emotional changes elicited by rarely presented facial expressions (deviants) among frequently presented facial expressions (standards). In this study, we investigated the impact of facial feedback on the automatic processing of facial expressions. For this purpose, participants (n = 19) performed a centrally presented visual detection task while neutral (standard), happy, and sad faces (deviants) were presented peripherally. During the task, facial feedback was manipulated by different pen holding conditions (holding the pen with teeth, lips, or nondominant hand). Our results indicate that automatic processing of facial expressions is influenced and thus dependent on the own facial feedback.

A neuromarker of clinical outcome in attention bias modification therapy for social anxiety disorder

BACKGROUND

Attention bias modification (ABM) therapy aims to modify threat-related attention patterns via computerized tasks. Despite showing medium clinical effect sizes for anxiety disorders, underlying neural-cognitive mechanisms of change remain unclear. We used visual mismatch negativity (vMMN), an event-related potential sensitive to violations of learned statistical contingencies, to assess therapy-related contingency extraction processes in healthy participants and in patients with social anxiety disorder (SAD). We then assessed whether vMMN amplitude predicts ABM treatment outcome.

METHODS

A modified version of the dot-probe task was used to elicit vMMN, in which 80% of trials were standard and 20% were deviant. In study 1, 30 healthy adults were randomly assigned to one of two ABM conditions: one in which threat-congruent targets were deviant trials and threat-incongruent targets were standard trials, and another in which the contingency was reversed. Electroencephalography (EEG) was continuously measured and vMMN analyzed. In study 2, 38 patients with SAD underwent six sessions of ABM therapy. We tested whether rule extraction in the ABM task, indicated by vMMN amplitude, predicts treatment outcome.

RESULTS

vMMN clearly emerged over prespecified scalp locations indicating contingency extraction during ABM (study 1). vMMN amplitude predicted clinical improvement after ABM therapy, uniquely accounting for 7% and 14.4% of the variance in clinician-rated and self-reported posttreatment SAD symptoms, respectively.

CONCLUSIONS

vMMN emerges as a neural marker for contingency learning in ABM, suggesting a significant role for contingency extraction processes in the clinical efficacy of this therapy.

Detecting differences with magnetoencephalography of somatosensory processing after tactile and electrical stimuli

BACKGROUND

Deviant stimuli within a standard, frequent stimulus train induce a cortical somatosensory mismatch response (SMMR). The SMMR reflects the brain's automatic mechanism for the detection of change in a somatosensory domain. It is usually elicited by electrical stimulation, which activates nerve fibers and receptors in superficial and deep skin layers, whereas tactile stimulation is closer to natural stimulation and activates uniform fiber types. We recorded SMMRs after electrical and tactile stimuli.

METHOD

306-channel magnetoencephalography recordings were made with 16 healthy adults under two conditions: electrical (eSMMR) and tactile (tSMMR) stimulations. The SMMR protocol consisted of 1000 stimuli with 10% deviants to fingers.

RESULTS

Sensor-level analysis revealed stronger activation after deviant stimulation in bilateral channel locations approximately corresponding to parietal cortical areas within both stimulation conditions. Between conditions, deviant tSMMR showed stronger activation in the ipsilateral channels. Based on sensor-level results, two components, M50 and SMMR (40-58 and 110-185 ms), were compared at the source-level. Deviant stimulation elicited stronger contralateral SI activation during M50 component in both conditions. SMMR was observed with both conditions, activating contralateral SII after deviant stimulation. However, only tSMMR showed long latency activation in bilateral SI cortices. This suggests that there is an integration of both body sides during the automatic stages of tactile processing in SI cortices.

CONCLUSIONS

This study indicates that tactile stimulation (tSMMR) is a feasible method for investigating the brain's mechanism for detecting somatosensory changes; this may extend the clinical utility of tSMMR for assessing disorders involving altered somatosensory processing.

The dysfunction of processing emotional faces in schizophrenia revealed by expression-related visual mismatch negativity

To investigate the emotional face processing in patients with schizophrenia, the preattentive automatic processing of emotional faces in individuals with schizophrenia was compared with that of age-matched healthy control group as indexed by the expressional mismatch negativity (EMMN) elicited by facial expressions. Compared with neutral faces as standard stimuli, deviant emotional faces elicited posterior EMMN between 150 and 500 ms after stimuli onset, with larger amplitudes for sad than happy deviant faces. Both early and late EMMNs significantly decreased in the schizophrenia group, regardless of sad or happy EMMN, in comparison with the healthy control group. These data suggest the dysfunction of automatic processing of expressional information in patients with schizophrenia.

Emotional context restores cortical prediction error responses in schizophrenia

The mismatch negativity (MMN) deficit in schizophrenia is a consistently replicated finding and is considered a potential biomarker. From the cognitive neuroscience perspective, MMN represents a cortical correlate of the prediction error, a fundamental computational operator that may be at the core of various cognitive and clinical deficits observed in schizophrenia. The impact of emotion on cognitive processes in schizophrenia is insufficiently understood, and its impact on basic operators of cortical computation is largely unknown. In the visual domain, the facial expression mismatch negativity (EMMN) offers an opportunity to investigate basic computational operators in purely cognitive and in emotional contexts. In this study, we asked whether emotional context enhances cortical prediction error responses in patients with schizophrenia, as is the case in normal subjects. Therefore, seventeen patients with schizophrenia and eighteen controls completed a visual sequence oddball task, which allows for directly comparing MMN components evoked by deviants with high, intermediate and low emotional engagement. Interestingly, patients with schizophrenia showed pronounced deficits in response to neutral stimuli, but almost normal responses to emotional stimuli. The dissociation between impaired MMN and normal EMMN suggests that emotional context not only enhances, but restores cortical prediction error responses in patients with schizophrenia to near-normal levels. Our results show that emotional processing in schizophrenia is not necessarily defect; more likely, emotional processing heterogeneously impacts on cognition in schizophrenia. In fact, this study suggests that emotional context may even compensate for cognitive deficits in schizophrenia that are, in a different sensory domain, discussed as biomarkers.

Principles of Temporal Processing Across the Cortical Hierarchy

The world is richly structured on multiple spatiotemporal scales. In order to represent spatial structure, many machine-learning models repeat a set of basic operations at each layer of a hierarchical architecture. These iterated spatial operations - including pooling, normalization and pattern completion - enable these systems to recognize and predict spatial structure, while robust to changes in the spatial scale, contrast and noisiness of the input signal. Because our brains also process temporal information that is rich and occurs across multiple time scales, might the brain employ an analogous set of operations for temporal information processing? Here we define a candidate set of temporal operations, and we review evidence that they are implemented in the mammalian cerebral cortex in a hierarchical manner. We conclude that multiple consecutive stages of cortical processing can be understood to perform temporal pooling, temporal normalization and temporal pattern completion.