Mismatch negativity: deviance detection or the maintenance of the 'standard'

Markov chains as a proxy for the predictive memory representations underlying mismatch negativity

Events not conforming to a regularity inherent to a sequence of events elicit prediction error signals of the brain such as the Mismatch Negativity (MMN) and impair behavioral task performance. Events conforming to a regularity lead to attenuation of brain activity such as stimulus-specific adaptation (SSA) and behavioral benefits. Such findings are usually explained by theories stating that the information processing system predicts the forthcoming event of the sequence via detected sequential regularities. A mathematical model that is widely used to describe, to analyze and to generate event sequences are Markov chains: They contain a set of possible events and a set of probabilities for transitions between these events (transition matrix) that allow to predict the next event on the basis of the current event and the transition probabilities. The accuracy of such a prediction depends on the distribution of the transition probabilities. We argue that Markov chains also have useful applications when studying cognitive brain functions. The transition matrix can be regarded as a proxy for generative memory representations that the brain uses to predict the next event. We assume that detected regularities in a sequence of events correspond to (a subset of) the entries in the transition matrix. We apply this idea to the Mismatch Negativity (MMN) research and examine three types of MMN paradigms: classical oddball paradigms emphasizing sound probabilities, between-sound regularity paradigms manipulating transition probabilities between adjacent sounds, and action-sound coupling paradigms in which sounds are associated with actions and their intended effects. We show that the Markovian view on MMN yields theoretically relevant insights into the brain processes underlying MMN and stimulates experimental designs to study the brain's processing of event sequences.

Habituation, Adaptation and Prediction Processes in Neurodevelopmental Disorders: A Comprehensive Review

Habituation, the simplest form of learning preserved across species and evolution, is characterized by a response decrease as a stimulus is repeated. This adaptive function has been shown to be altered in some psychiatric and neurodevelopmental disorders such as autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD) or schizophrenia. At the brain level, habituation is characterized by a decrease in neural activity as a stimulation is repeated, referred to as neural adaptation. This phenomenon influences the ability to make predictions and to detect change, two processes altered in some neurodevelopmental and psychiatric disorders. In this comprehensive review, the objectives are to characterize habituation, neural adaptation, and prediction throughout typical development and in neurodevelopmental disorders; and to evaluate their implication in symptomatology, specifically in sensitivity to change or need for sameness. A summary of the different approaches to investigate adaptation will be proposed, in which we report the contribution of animal studies as well as electrophysiological studies in humans to understanding of underlying neuronal mechanisms.

Pre-attentive representation of prediction certainty in autism: A mismatch negativity (MMN) study

According to predictive processing theories of perception, the brain generates predictions to prepare for sensory input, and calibrates certainty of predictions based on their likelihood. When an input doesn't match the prediction, an error signal leads to updating of the predictive model. Prior research suggests altered prediction certainty in autism, but predictive processing occurs across the cortical hierarchy, and the stage(s) of processing where prediction certainty breaks down is unknown. We therefore tested the integrity of prediction certainty in autism at pre-attentive and relatively automatic processing stages using the pre-attentive Mismatch Negativity (MMN) brain response. The MMN occurs in response to a "deviant" presented in a stream of "standards" and is measured while the participant performs an orthogonal task. Most critically, MMN amplitude typically varies with the level of certainty associated with the prediction. We recorded high-density EEG while presenting adolescents and young adults with and without autism with repetitive tones every half second (the standard) interspersed with infrequent pitch and inter-stimulus-interval (ISI) deviants. Pitch and ISI deviant probabilities were manipulated at 4, 8, or 16% within a block of trials to test whether MMN amplitude varied in a typical manner with respect to probability. For both groups, Pitch-MMN amplitude increased as the probability of deviance decreased. Unexpectedly, ISI-MMN amplitude did not reliably vary by probability in either group. Our Pitch-MMN findings suggest intact neural representation of pre-attentive prediction certainty in autism, addressing a critical knowledge gap in autism research. The implications of these findings are considered.

LAY SUMMARY

Our brains are always trying to predict what will happen next. For example, when you open your utensil drawer, it would be surprising to see books because your brain expected to see utensils. In our study, we looked at whether the brains of autistic individuals automatically and accurately recognize when something unexpected happens. Results showed similar brain patterns in individuals with and without autism, suggesting that responses to prediction violations are generated in a typical manner during early cortical information processing.

Early auditory processing dysfunction in schizophrenia: Mechanisms and implications

Schizophrenia is a major mental disorder that affects approximately 1% of the population worldwide. Cognitive deficits are a key feature of the disorder and a primary cause of long-term disability. Over the past decades, significant literature has accumulated demonstrating impairments in early auditory perceptual processes in schizophrenia. In this review, we first describe early auditory dysfunction in schizophrenia from both a behavioral and neurophysiological perspective and examine their interrelationship with both higher order cognitive constructs and social cognitive processes. Then, we provide insights into underlying pathological processes, especially in relationship to glutamatergic and N-methyl-D-aspartate receptor (NMDAR) dysfunction models. Finally, we discuss the utility of early auditory measures as both treatment targets for precision intervention and as translational biomarkers for etiological investigation. Altogether, this review points out the crucial role of early auditory deficits in the pathophysiology of schizophrenia, in addition to major implications for early intervention and auditory-targeted approaches.

Auditory representations for long lasting sounds: Insights from event-related brain potentials and neural oscillations

The basic features of short sounds, such as frequency and intensity including their temporal dynamics, are integrated in a unitary representation. Knowledge on how our brain processes long lasting sounds is scarce. We review research utilizing the Mismatch Negativity event-related potential and neural oscillatory activity for studying representations for long lasting simple versus complex sounds such as sinusoidal tones versus speech. There is evidence for a temporal constraint in the formation of auditory representations: Auditory edges like sound onsets within long lasting sounds open a temporal window of about 350 ms in which the sounds' dynamics are integrated into a representation, while information beyond that window contributes less to that representation. This integration window segments the auditory input into short chunks. We argue that the representations established in adjacent integration windows can be concatenated into an auditory representation of a long sound, thus, overcoming the temporal constraint.

Predictability-Based Source Segregation and Sensory Deviance Detection in Auditory Aging

When multiple sound sources are present at the same time, auditory perception is often challenged with disentangling the resulting mixture and focusing attention on the target source. It has been repeatedly demonstrated that background (distractor) sound sources are easier to ignore when their spectrotemporal signature is predictable. Prior evidence suggests that this ability to exploit predictability for foreground-background segregation degrades with age. On a theoretical level, this has been related with an impairment in elderly adults’ capabilities to detect certain types of sensory deviance in unattended sound sequences. Yet the link between those two capacities, deviance detection and predictability-based sound source segregation, has not been empirically demonstrated. Here we report on a combined behavioral-EEG study investigating the ability of elderly listeners (60–75 years of age) to use predictability as a cue for sound source segregation, as well as their sensory deviance detection capacities. Listeners performed a detection task on a target stream that can only be solved when a concurrent distractor stream is successfully ignored. We contrast two conditions whose distractor streams differ in their predictability. The ability to benefit from predictability was operationalized as performance difference between the two conditions. Results show that elderly listeners can use predictability for sound source segregation at group level, yet with a high degree of inter-individual variation in this ability. In a further, passive-listening control condition, we measured correlates of deviance detection in the event-related brain potential (ERP) elicited by occasional deviations from the same spectrotemporal pattern as used for the predictable distractor sequence during the behavioral task. ERP results confirmed neural signatures of deviance detection in terms of mismatch negativity (MMN) at group level. Correlation analyses at single-subject level provide no evidence for the hypothesis that deviance detection ability (measured by MMN amplitude) is related to the ability to benefit from predictability for sound source segregation. These results are discussed in the frameworks of sensory deviance detection and predictive coding.

Standard Tone Stability as a Manipulation of Precision in the Oddball Paradigm: Modulation of Prediction Error Responses to Fixed-Probability Deviants

Electrophysiological sensory deviance detection signals, such as the mismatch negativity (MMN), have been interpreted from the predictive coding framework as manifestations of prediction error (PE). From a frequentist perspective of the classic oddball paradigm, deviant stimuli are unexpected because of their low probability. However, the amount of PE elicited by a stimulus can be dissociated from its probability of occurrence: when the observer cannot make confident predictions, any event holds little surprise value, no matter how improbable. Here we tested the hypothesis that the magnitude of the neural response elicited to an improbable sound (D) would scale with the precision of the prediction derived from the repetition of another sound (S), by manipulating repetition stability. We recorded the Electroencephalogram (EEG) from 20 participants while passively listening to 4 types of isochronous pure tone sequences differing in the probability of the S tone (880 Hz) while holding constant the probability of the D tone [1,046 Hz; p(D) = 1/11]: Oddball [p(S) = 10/11]; High confidence (7/11); Low confidence (4/11); and Random (1/11). Tones of 9 different frequencies were equiprobably presented as fillers [p(S) + p(D) + p(F) = 1]. Using a mass-univariate non-parametric, cluster-based correlation analysis controlling for multiple comparisons, we found that the amplitude of the deviant-elicited ERP became more negative with increasing S probability, in a time-electrode window consistent with the MMN (ca. 120–200 ms; frontal), suggesting that the strength of a PE elicited to an improbable event indeed increases with the precision of the predictive model.

Pre-attentive auditory change detection for rapid auditory transient combinations: Insight from age-related processing changes

The N1 event-related potential (ERP) enhancement to auditory transients preceded briefly by another transient has been interpreted as a reflection of latent inhibition, or alternatively, as a superimposing mismatch negativity (MMN) to rare transient event combinations. In a previous study (Volosin, Gaál, & Horváth, 2017a), when rare glides preceded frequent gaps by 150 ms in continuous tones, gap-related N1 was enhanced in younger adults while P2 was attenuated both in younger and older adults, which could be parsimoniously explained by MMN overlap which was delayed with aging. The present study replicated and extended these results with a condition in which the roles of the two event types were reversed. Transients separated by 150 ms elicited delayed MMN in older adults, supporting the MMN interpretation over the latent inhibition account. Furthermore, the divergence of N1 and MMN elicitation patterns demonstrated the independence of N1 and MMN.

Emergence of prediction error along the human auditory hierarchy

Auditory prediction errors have been extensively associated with the mismatch negativity (MMN), a cortical auditory evoked potential that denotes deviance detection. Yet, many studies lacked the appropriate controls to disentangle sensory adaptation from prediction error. Furthermore, subcortical deviance detection has been shown in humans through recordings of the frequency-following response (FFR), an early auditory evoked potential that reflects the neural tracking of the periodic characteristics of a sound, suggesting the possibility that prediction errors emerge subcortically in the auditory pathway. The present study aimed at investigating the emergence of prediction error along the auditory hierarchy in humans through combined recordings of the FFR and the MMN, tapping at subcortical and cortical levels, respectively, while disentangling prediction error from sensory adaptation with the use of appropriate controls. "Oddball" sequences of pure tones featuring repeated "standard" stimuli (269 Hz; p = 0.8) and rare "deviant" stimuli (p = 0.2; of 289, 329 and 409 Hz delivered in separated blocks to test "frequency separation" effects) were presented to nineteen healthy young participants. "Reversed" oddball sequences (where standard and deviant tones swapped their roles) were presented allowing comparison of responses to same physical stimuli as a function of functional role (i.e., standard, deviant). Critically, control sequences featuring five equiprobable tones (p = 0.2) allowed to dissociate sensory adaptation from prediction error. Results revealed that the MMN amplitude increased as a function of frequency separation yet displayed the same amplitude when retrieved against the control sequences, confirming previous results. FFRs showed repetition enhancement effects across all frequency separations, as supported by larger spectral amplitude to standard than to deviant and control stimuli. This pattern of results provides insights into the hierarchy of the human prediction error system in audition, suggesting that prediction errors in humans emerge at cortical level.

Making Sense of Mismatch Negativity

Evoked potentials provide valuable insight into brain processes that are integral to our ability to interact effectively and efficiently in the world. The mismatch negativity (MMN) component of the evoked potential has proven highly informative on the ways in which sensitivity to regularity contributes to perception and cognition. This review offers a compendium of research on MMN with a view to scaffolding an appreciation for its use as a tool to explore the way regularities contribute to predictions about the sensory environment over many timescales. In compiling this work, interest in MMN as an index of sensory encoding and memory are addressed, as well as attention. Perspectives on the possible underlying computational processes are reviewed as well as recent observations that invite consideration of how MMN relates to how we learn, what we learn, and why.

Visual mismatch negativity and stimulus-specific adaptation: the role of stimulus complexity

The present study investigated the function of the brain activity underlying the visual mismatch negativity (vMMN) event-related potential (ERP) component. Snowflake patterns (complex stimuli) were presented as deviants and oblique bar patterns (simple stimuli) as standards, and vice versa in a passive oddball paradigm. Control (equiprobable) sequences of either complex shape patterns or oblique bar patterns with various orientations were also presented. VMMN appeared as the difference between the ERP to the oddball deviant and the ERP to the control (deviant minus control ERP difference). Apart from the shorter latency of the vMMN to the oblique bar pattern as deviant, vMMN to both deviants was similar, i.e., there was no amplitude difference. We attributed the function of the brain processes underlying vMMN to the detection of the infrequent stimulus type (also represented in memory) instead of a call for further processing (a possibility for acquiring more precise representation) of the deviant. An unexpected larger adaptation (control minus standard ERP difference) to the snowflake pattern was also obtained. We suggest that this was due to the acquisition of a more elaborate memory representation of the more complex stimulus.

Neural mechanisms for coping with acoustically reduced speech

In spoken language, reductions of word forms occur regularly and need to be accommodated by the listener. Intriguingly, this accommodation is usually achieved without any apparent effort. The neural bases of this cognitive skill are not yet fully understood. We here presented participants with reduced words that were either preceded by a related or an unrelated visual prime and compared electric brain responses to reduced words with those to their full counterparts. In time-domain, we found a positivity between 400 and 600 ms differing between reduced and full forms. A later positivity distinguished primed and unprimed words and was modulated by reduction. In frequency-domain, alpha suppression was stronger for reduced than for full words. The time- and frequency-domain reduction effects converge towards the view that reduced words draw on attention and memory mechanisms. Our data demonstrate the importance of interactive processing of bottom-up and top-down information for the comprehension of reduced words.

The Neuronal Basis of Predictive Coding Along the Auditory Pathway: From the Subcortical Roots to Cortical Deviance Detection

In this review, we attempt to integrate the empirical evidence regarding stimulus-specific adaptation (SSA) and mismatch negativity (MMN) under a predictive coding perspective (also known as Bayesian or hierarchical-inference model). We propose a renewed methodology for SSA study, which enables a further decomposition of deviance detection into repetition suppression and prediction error, thanks to the use of two controls previously introduced in MMN research: the many-standards and the cascade sequences. Focusing on data obtained with cellular recordings, we explain how deviance detection and prediction error are generated throughout hierarchical levels of processing, following two vectors of increasing computational complexity and abstraction along the auditory neuraxis: from subcortical toward cortical stations and from lemniscal toward nonlemniscal divisions. Then, we delve into the particular characteristics and contributions of subcortical and cortical structures to this generative mechanism of hierarchical inference, analyzing what is known about the role of neuromodulation and local microcircuitry in the emergence of mismatch signals. Finally, we describe how SSA and MMN are occurring at similar time frame and cortical locations, and both are affected by the manipulation of N-methyl- D-aspartate receptors. We conclude that there is enough empirical evidence to consider SSA and MMN, respectively, as the microscopic and macroscopic manifestations of the same physiological mechanism of deviance detection in the auditory cortex. Hence, the development of a common theoretical framework for SSA and MMN is all the more recommendable for future studies. In this regard, we suggest a shared nomenclature based on the predictive coding interpretation of deviance detection.

Great Expectations: Is there Evidence for Predictive Coding in Auditory Cortex?

Predictive coding is possibly one of the most influential, comprehensive, and controversial theories of neural function. While proponents praise its explanatory potential, critics object that key tenets of the theory are untested or even untestable. The present article critically examines existing evidence for predictive coding in the auditory modality. Specifically, we identify five key assumptions of the theory and evaluate each in the light of animal, human and modeling studies of auditory pattern processing. For the first two assumptions - that neural responses are shaped by expectations and that these expectations are hierarchically organized - animal and human studies provide compelling evidence. The anticipatory, predictive nature of these expectations also enjoys empirical support, especially from studies on unexpected stimulus omission. However, for the existence of separate error and prediction neurons, a key assumption of the theory, evidence is lacking. More work exists on the proposed oscillatory signatures of predictive coding, and on the relation between attention and precision. However, results on these latter two assumptions are mixed or contradictory. Looking to the future, more collaboration between human and animal studies, aided by model-based analyses will be needed to test specific assumptions and implementations of predictive coding - and, as such, help determine whether this popular grand theory can fulfill its expectations.

Differential neural mechanisms for early and late prediction error detection

Emerging evidence indicates that prediction, instantiated at different perceptual levels, facilitate visual processing and enable prompt and appropriate reactions. Until now, the mechanisms underlying the effect of predictive coding at different stages of visual processing have still remained unclear. Here, we aimed to investigate early and late processing of spatial prediction violation by performing combined recordings of saccadic eye movements and fast event-related fMRI during a continuous visual detection task. Psychophysical reverse correlation analysis revealed that the degree of mismatch between current perceptual input and prior expectations is mainly processed at late rather than early stage, which is instead responsible for fast but general prediction error detection. Furthermore, our results suggest that conscious late detection of deviant stimuli is elicited by the assessment of prediction error’s extent more than by prediction error per se. Functional MRI and functional connectivity data analyses indicated that higher-level brain systems interactions modulate conscious detection of prediction error through top-down processes for the analysis of its representational content, and possibly regulate subsequent adaptation of predictive models. Overall, our experimental paradigm allowed to dissect explicit from implicit behavioral and neural responses to deviant stimuli in terms of their reliance on predictive models.

On the development of auditory distraction: A review

The present review focuses on the development of involuntary attention mechanisms in the context of the occurrence of unexpected events during childhood. We introduce a prevailing three-stage model of auditory involuntary attention describing the processes leading to, accompanying, and following the distraction of attention by prediction violations: (a) the automatic detection of prediction violations (associated with the event-related potential [ERP] component mismatch negativity [MMN]), (b) the involuntary orienting of attention processes towards the prediction violating sound (associated with the ERP component P3a), and (c) the reorienting back to task-relevant information (associated with the ERP components reorienting negativity [RON] or late discriminative negativity [LDN]). Within this framework we give an overview of studies investigating MMN, P3a, RON/LDN, and behavioral distraction effects in children. We discuss the development of the underlying involuntary attention mechanisms and highlight the relevance of and future perspectives for this important field of research.

Auditory perceptual objects as generative models: Setting the stage for communication by sound

Communication by sounds requires that the communication channels (i.e. speech/speakers and other sound sources) had been established. This allows to separate concurrently active sound sources, to track their identity, to assess the type of message arriving from them, and to decide whether and when to react (e.g., reply to the message). We propose that these functions rely on a common generative model of the auditory environment. This model predicts upcoming sounds on the basis of representations describing temporal/sequential regularities. Predictions help to identify the continuation of the previously discovered sound sources to detect the emergence of new sources as well as changes in the behavior of the known ones. It produces auditory event representations which provide a full sensory description of the sounds, including their relation to the auditory context and the current goals of the organism. Event representations can be consciously perceived and serve as objects in various cognitive operations.

Noise occlusion in discrete tone sequences as a tool towards auditory predictive processing?

The notion of predictive coding is a common feature of many theories of auditory information processing. Experimental demonstrations of predictive auditory processing often rest on omitting predictable input in order to uncover the prediction made by the brain. Findings show that auditory cortical activity elicited by the omission of a predictable tone resembles the activity elicited by the actual tone. Here we attempted to extend this approach towards using noises instead of omissions in order to capture a more prevalent case of degraded sensory input. By applying a subtraction approach to remove ERP effects of the noise itself, auditory cortical activity elicited "behind" the noise was uncovered. We hypothesized that ERPs elicited behind noise stimuli covering predictable tones should be more similar to ERPs elicited by the actual tones than when the same comparison is made for unpredictable tones. ERP results during passive listening partly confirm this hypothesis, but also point towards some methodological caveats in this particular approach towards studying neural correlates of predictive auditory processing due to contributions from predictability-unrelated factors. A follow-up active listening condition indicated that participants were not more likely to perceive the tone sequence as continuous when a predictable tone was covered with noise than when this pertained to an unpredictable tone. Overall, the noise-based paradigm in its present form was not shown to be successful in revealing predictive processing in perceptual judgments or early neural correlates of sound processing. We discuss these findings in the contexts of predictive processing and illusory auditory continuity. This article is part of a Special Issue entitled SI: Prediction and Attention.

How does the extraction of local and global auditory regularities vary with context?

How does the human brain extract regularities from its environment? There is evidence that short range or ‘local’ regularities (within seconds) are automatically detected by the brain while long range or ‘global’ regularities (over tens of seconds or more) require conscious awareness. In the present experiment, we asked whether participants' attention was needed to acquire such auditory regularities, to detect their violation or both. We designed a paradigm in which participants listened to predictable sounds. Subjects could be distracted by a visual task at two moments: when they were first exposed to a regularity or when they detected violations of this regularity. MEG recordings revealed that early brain responses (100–130 ms) to violations of short range regularities were unaffected by visual distraction and driven essentially by local transitional probabilities. Based on global workspace theory and prior results, we expected that visual distraction would eliminate the long range global effect, but unexpectedly, we found the contrary, i.e. late brain responses (300–600 ms) to violations of long range regularities on audio-visual trials but not on auditory only trials. Further analyses showed that, in fact, visual distraction was incomplete and that auditory and visual stimuli interfered in both directions. Our results show that conscious, attentive subjects can learn the long range dependencies present in auditory stimuli even while performing a visual task on synchronous visual stimuli. Furthermore, they acquire a complex regularity and end up making different predictions for the very same stimulus depending on the context (i.e. absence or presence of visual stimuli). These results suggest that while short-range regularity detection is driven by local transitional probabilities between stimuli, the human brain detects and stores long-range regularities in a highly flexible, context dependent manner.

Mismatch negativity of sad syllables is absent in patients with major depressive disorder

BACKGROUND

Major depressive disorder (MDD) is an important and highly prevalent mental disorder characterized by anhedonia and a lack of interest in everyday activities. Additionally, patients with MDD appear to have deficits in various cognitive abilities. Although a number of studies investigating the central auditory processing of low-level sound features in patients with MDD have demonstrated that this population exhibits impairments in automatic processing, the influence of emotional voice processing has yet to be addressed. To explore the automatic processing of emotional prosodies in patients with MDD, we analyzed the ability to detect automatic changes using event-related potentials (ERPs).

METHOD

This study included 18 patients with MDD and 22 age- and sex-matched healthy controls. Subjects were instructed to watch a silent movie but to ignore the afferent acoustic emotional prosodies presented to both ears while continuous electroencephalographic activity was synchronously recorded. Prosodies included meaningless syllables, such as "dada" spoken with happy, angry, sad, or neutral tones. The mean amplitudes of the ERPs elicited by emotional stimuli and the peak latency of the emotional differential waveforms were analyzed.

RESULTS

The sad MMN was absent in patients with MDD, whereas the happy and angry MMN components were similar across groups. The abnormal sad emotional MMN component was not significantly correlated with the HRSD-17 and HAMA scores, respectively.

CONCLUSION

The data indicate that patients with MDD are impaired in their ability to automatically process sad prosody, whereas their ability to process happy and angry prosodies remains normal. The dysfunctional sad emotion-related MMN in patients with MDD were not correlated with depression symptoms. The blunted MMN of sad prosodies could be considered a trait of MDD.

Attention effects on auditory scene analysis: insights from event-related brain potentials

Sounds emitted by different sources arrive at our ears as a mixture that must be disentangled before meaningful information can be retrieved. It is still a matter of debate whether this decomposition happens automatically or requires the listener's attention. These opposite positions partly stem from different methodological approaches to the problem. We propose an integrative approach that combines the logic of previous measurements targeting either auditory stream segregation (interpreting a mixture as coming from two separate sources) or integration (interpreting a mixture as originating from only one source). By means of combined behavioral and event-related potential (ERP) measures, our paradigm has the potential to measure stream segregation and integration at the same time, providing the opportunity to obtain positive evidence of either one. This reduces the reliance on zero findings (i.e., the occurrence of stream integration in a given condition can be demonstrated directly, rather than indirectly based on the absence of empirical evidence for stream segregation, and vice versa). With this two-way approach, we systematically manipulate attention devoted to the auditory stimuli (by varying their task relevance) and to their underlying structure (by delivering perceptual tasks that require segregated or integrated percepts). ERP results based on the mismatch negativity (MMN) show no evidence for a modulation of stream integration by attention, while stream segregation results were less clear due to overlapping attention-related components in the MMN latency range. We suggest future studies combining the proposed two-way approach with some improvements in the ERP measurement of sequential stream segregation.

Prediction in the service of comprehension: modulated early brain responses to omitted speech segments

Speech signals are often compromised by disruptions originating from external (e.g., masking noise) or internal (e.g., inaccurate articulation) sources. Speech comprehension thus entails detecting and replacing missing information based on predictive and restorative neural mechanisms. The present study targets predictive mechanisms by investigating the influence of a speech segment's predictability on early, modality-specific electrophysiological responses to this segment's omission. Predictability was manipulated in simple physical terms in a single-word framework (Experiment 1) or in more complex semantic terms in a sentence framework (Experiment 2). In both experiments, final consonants of the German words Lachs ([laks], salmon) or Latz ([lats], bib) were occasionally omitted, resulting in the syllable La ([la], no semantic meaning), while brain responses were measured with multi-channel electroencephalography (EEG). In both experiments, the occasional presentation of the fragment La elicited a larger omission response when the final speech segment had been predictable. The omission response occurred ∼125-165 msec after the expected onset of the final segment and showed characteristics of the omission mismatch negativity (MMN), with generators in auditory cortical areas. Suggestive of a general auditory predictive mechanism at work, this main observation was robust against varying source of predictive information or attentional allocation, differing between the two experiments. Source localization further suggested the omission response enhancement by predictability to emerge from left superior temporal gyrus and left angular gyrus in both experiments, with additional experiment-specific contributions. These results are consistent with the existence of predictive coding mechanisms in the central auditory system, and suggestive of the general predictive properties of the auditory system to support spoken word recognition.

Modelling trial-by-trial changes in the mismatch negativity

The mismatch negativity (MMN) is a differential brain response to violations of learned regularities. It has been used to demonstrate that the brain learns the statistical structure of its environment and predicts future sensory inputs. However, the algorithmic nature of these computations and the underlying neurobiological implementation remain controversial. This article introduces a mathematical framework with which competing ideas about the computational quantities indexed by MMN responses can be formalized and tested against single-trial EEG data. This framework was applied to five major theories of the MMN, comparing their ability to explain trial-by-trial changes in MMN amplitude. Three of these theories (predictive coding, model adjustment, and novelty detection) were formalized by linking the MMN to different manifestations of the same computational mechanism: approximate Bayesian inference according to the free-energy principle. We thereby propose a unifying view on three distinct theories of the MMN. The relative plausibility of each theory was assessed against empirical single-trial MMN amplitudes acquired from eight healthy volunteers in a roving oddball experiment. Models based on the free-energy principle provided more plausible explanations of trial-by-trial changes in MMN amplitude than models representing the two more traditional theories (change detection and adaptation). Our results suggest that the MMN reflects approximate Bayesian learning of sensory regularities, and that the MMN-generating process adjusts a probabilistic model of the environment according to prediction errors.

The ability to predict one's environment is crucial for adaptive and proactive behaviour. It requires learning a mental model that captures the environment's statistical regularities. A process of this sort is thought to be reflected by the mismatch negativity (MMN) potential, a non-invasive electrophysiological measure of the neural response to regularity violation by sensory stimuli. However, the exact computational processes reflected by the MMN remain a matter of debate. We developed a modelling framework in which competing hypotheses about these processes can be objectively compared by their ability to predict single-trial MMN amplitudes. We applied this framework to formalize five major MMN theories and propose a unifying view on three distinct theories which explain the MMN as a reflection of prediction errors, model adjustment, and novelty detection, respectively. We assessed our models of the five theories with EEG data from eight healthy volunteers. Our results are consistent with the idea that the MMN arises from prediction error driven adjustments of a probabilistic mental model of the environment.

Detection of simple and pattern regularity violations occurs at different levels of the auditory hierarchy

Auditory deviance detection in humans is indexed by the mismatch negativity (MMN), a component of the auditory evoked potential (AEP) of the electroencephalogram (EEG) occurring at a latency of 100–250 ms after stimulus onset. However, by using classic oddball paradigms, differential responses to regularity violations of simple auditory features have been found at the level of the middle latency response (MLR) of the AEP occurring within the first 50 ms after stimulus (deviation) onset. These findings suggest the existence of fast deviance detection mechanisms for simple feature changes, but it is not clear whether deviance detection among more complex acoustic regularities could be observed at such early latencies. To test this, we examined the pre-attentive processing of rare stimulus repetitions in a sequence of tones alternating in frequency in both long and middle latency ranges. Additionally, we introduced occasional changes in the interaural time difference (ITD), so that a simple-feature regularity could be examined in the same paradigm. MMN was obtained for both repetition and ITD deviants, occurring at 150 ms and 100 ms after stimulus onset respectively. At the level of the MLR, a difference was observed between standards and ITD deviants at the Na component (20–30 ms after stimulus onset), for 800 Hz tones, but not for repetition deviants. These findings suggest that detection mechanisms for deviants to simple regularities, but not to more complex regularities, are already activated in the MLR range, supporting the view that the auditory deviance detection system is organized in a hierarchical manner.

A sparse neural code for some speech sounds but not for others

The precise neural mechanisms underlying speech sound representations are still a matter of debate. Proponents of 'sparse representations' assume that on the level of speech sounds, only contrastive or otherwise not predictable information is stored in long-term memory. Here, in a passive oddball paradigm, we challenge the neural foundations of such a 'sparse' representation; we use words that differ only in their penultimate consonant ("coronal" [t] vs. "dorsal" [k] place of articulation) and for example distinguish between the German nouns Latz ([lats]; bib) and Lachs ([laks]; salmon). Changes from standard [t] to deviant [k] and vice versa elicited a discernible Mismatch Negativity (MMN) response. Crucially, however, the MMN for the deviant [lats] was stronger than the MMN for the deviant [laks]. Source localization showed this difference to be due to enhanced brain activity in right superior temporal cortex. These findings reflect a difference in phonological 'sparsity': Coronal [t] segments, but not dorsal [k] segments, are based on more sparse representations and elicit less specific neural predictions; sensory deviations from this prediction are more readily 'tolerated' and accordingly trigger weaker MMNs. The results foster the neurocomputational reality of 'representationally sparse' models of speech perception that are compatible with more general predictive mechanisms in auditory perception.

Regularity extraction from non-adjacent sounds

The regular behavior of sound sources helps us to make sense of the auditory environment. Regular patterns may, for instance, convey information on the identity of a sound source (such as the acoustic signature of a train moving on the rails). Yet typically, this signature overlaps in time with signals emitted from other sound sources. It is generally assumed that auditory regularity extraction cannot operate upon this mixture of signals because it only finds regularities between adjacent sounds. In this view, the auditory environment would be grouped into separate entities by means of readily available acoustic cues such as separation in frequency and location. Regularity extraction processes would then operate upon the resulting groups. Our new experimental evidence challenges this view. We presented two interleaved sound sequences which overlapped in frequency range and shared all acoustic parameters. The sequences only differed in their underlying regular patterns. We inserted deviants into one of the sequences to probe whether the regularity was extracted. In the first experiment, we found that these deviants elicited the mismatch negativity (MMN) component. Thus the auditory system was able to find the regularity between the non-adjacent sounds. Regularity extraction was not influenced by sequence cohesiveness as manipulated by the relative duration of tones and silent inter-tone-intervals. In the second experiment, we showed that a regularity connecting non-adjacent sounds was discovered only when the intervening sequence also contained a regular pattern, but not when the intervening sounds were randomly varying. This suggests that separate regular patterns are available to the auditory system as a cue for identifying signals coming from distinct sound sources. Thus auditory regularity extraction is not necessarily confined to a processing stage after initial sound grouping, but may precede grouping when other acoustic cues are unavailable.

Responses to deviants are modulated by subthreshold variability of the standard

Auditory mechanisms automatically detect both basic features of sounds and the rules governing their presentation. In the oddball paradigm, the auditory system detects the sameness (or no-variability) rule when the same reference tone is consistently repeated. We used two oddball protocols, the classical one with a fixed reference and a modified one with a jittered reference, to determine whether the auditory system can detect subthreshold violations of sameness. We found that the response to the repeated standard was not modified by the small jitter. However, the response to the frequency oddball was smaller under the jittered protocol, indicating hypersensitivity to sameness. The sensitivity to jitter was largest when the oddball deviated by 8%, was smaller for 40%, and disappeared at 100% deviation, indicating that sensitivity to sameness is context dependent; namely, it is scaled with respect to the overall range of stimuli.

Prediction, cognition and the brain

The term “predictive brain” depicts one of the most relevant concepts in cognitive neuroscience which emphasizes the importance of “looking into the future”, namely prediction, preparation, anticipation, prospection or expectations in various cognitive domains. Analogously, it has been suggested that predictive processing represents one of the fundamental principles of neural computations and that errors of prediction may be crucial for driving neural and cognitive processes as well as behavior. This review discusses research areas which have recognized the importance of prediction and introduces the relevant terminology and leading theories in the field in an attempt to abstract some generative mechanisms of predictive processing. Furthermore, we discuss the process of testing the validity of postulated expectations by matching these to the realized events and compare the subsequent processing of events which confirm to those which violate the initial predictions. We conclude by suggesting that, although a lot is known about this type of processing, there are still many open issues which need to be resolved before a unified theory of predictive processing can be postulated with regard to both cognitive and neural functioning.

Differences in processing violations of sequential and feature regularities as revealed by visual event-related brain potentials

Identifying novel or unexpected events which violate predictions based on the regularities extracted from our environment is crucially important for adaptive behavior. However, the exact dynamics of processing such events is not well understood. Furthermore, it is not known in which degree the process of deviant detection differs across contexts and how much it depends on the characteristics of deviant events. This issue was addressed by the present study which used event-related potentials (ERPs) in order to investigate the dynamics of identifying two types of deviants presented within the same visual setting. These events violated expectations based on two different types of information contained within each trial, either temporal order of stimulus presentation (sequential deviant) or physical attributes shared by the majority of individual stimuli (feature deviant). The obtained results indicate a certain degree of similarity in detecting two deviant types which, when task-relevant, both elicited N2 and P3b event-related potential components. However, significant differences across different stages of their processing were also identified. First, only feature, but not sequential deviants elicited an N1 enhancement. Furthermore, N2 and P3b responses elicited by sequential and feature deviants differed in their latency and topography and, in case of P3b, amplitude. Taken together, these results suggest that the dynamics of detecting different types of deviants strongly depends on the specific characteristics of such events. Furthermore, the identified differences in the topography of N2 and P3b indicate distinct mechanisms underlying several stages of their processing.

Neurophysiological measures of sensory registration, stimulus discrimination, and selection in schizophrenia patients

Cortical Neurophysiological event related potentials (ERPs) are multidimensional measures of information processing that are well suited to efficiently parse automatic and controlled components of cognition that span the range of deficits exhibited in schizophrenia patients. Components following a stimulus reflect the sequence of neural processes triggered by the stimulus, beginning with early automatic sensory processes and proceeding through controlled decision and response related processes. Previous studies employing ERP paradigms have reported deficits of information processing in schizophrenia across automatic through attention dependent processes including sensory registration (N1), automatic change detection (MMN), the orienting or covert shift of attention towards novel or infrequent stimuli (P3a), and attentional allocation following successful target detection processes (P3b). These automatic and attention dependent information components are beginning to be recognized as valid targets for intervention in the context of novel treatment development for schizophrenia and related neuropsychiatric disorders. In this review, we describe three extensively studied ERP components (N1, mismatch negativity, P300) that are consistently deficient in schizophrenia patients and may serve as genetic endophenotypes and as quantitative biological markers of response outcome.