Mismatch negativity: deviance detection or the maintenance of the 'standard'

Units of sound representation and temporal integration: a mismatch negativity study

Two acoustic events occurring successively within 200 ms are processed as a single event when the first event predicts the occurrence of the second, but are processed as two separate events when the two events can also occur independently of each other and thus the second event provides new information. However, if the two events are carried by the same stimulus, they are always processed as a single event. This was shown by studies using the mismatch negativity (MMN) event-related potential (ERP). The current study was aimed at investigating the acoustic parameters that determine the integration of successive events within the putative temporal window of integration (TWI). The results demonstrate that temporal grouping (achieved here by presenting sounds in pairs) of the acoustic events within the TWI creates strong unitization, which is not broken up by higher level contingencies of the sound sequence, such as the predictability of the second successive event.

The mismatch negativity (MMN) in basic research of central auditory processing: a review

In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.

MMN or no MMN: no magnitude of deviance effect on the MMN amplitude

Based on results showing that the "deviant-minus-standard" estimate of the mismatch negativity (MMN) amplitude increases with increasing amounts of deviance, it has been suggested that the MMN amplitude reflects the amount of difference between the neural representations of the standard and the deviant sound. However, the deviant-minus-standard waveform also includes an N1 difference. We tested the effects of the magnitude of deviance on MMN while minimizing this N1 confound. We found no significant magnitude of deviance effect on the genuine MMN amplitude. Thus we suggest that the average MMN amplitude does not reflect the difference between neural stimulus representations; rather it may index the percentage of detected deviants, each of which elicits an MMN response of uniform amplitude. These results are compatible with an explanation suggesting that MMN is involved in maintaining a neural representation of the auditory environment.

Interpreting the Mismatch Negativity

The widely accepted “memory-mismatch” interpretation of the mismatch negativity (MMN) event-related brain potential (ERP) suggests that an MMN is elicited when an acoustic event deviates from a memory record describing the immediate history of the sound sequence. The first variant of the memory-mismatch theory suggested that the memory underlying MMN generation was a strong auditory sensory memory trace, which encoded the repetitive standard sound. This “trace-mismatch” explanation of MMN has been primarily based on results obtained in the auditory oddball paradigm. However, in recent years, MMN has been observed in stimulus paradigms containing no frequently repeating sound. We now suggest a different variant of the memory-mismatch interpretation of MMN in order to provide a unified explanation of all MMN phenomena. The regularity-violation explanation of MMN assumes that the memory records retaining the history of auditory stimulation are regularity representations. These representations encode rules extracted from the regular intersound relationships, which are mapped to the concrete sound sequence by finely detailed auditory sensory information. Auditory events are compared with temporally aligned predictions drawn from the regularity representations (predictive models) and the observable MMN response reflects a process updating the representations of those detected regularities whose prediction was mismatched by the acoustic input. It is further suggested that the auditory deviance detection system serves to organize sound in the brain: The predictive models maintained by the MMN-generating process provide the basis of temporal grouping, a crucial step in the formation of auditory objects.

The role of predictive models in the formation of auditory streams

Sounds provide us with useful information about our environment which complements that provided by other senses, but also poses specific processing problems. How does the auditory system distentangle sounds from different sound sources? And what is it that allows intermittent sound events from the same source to be associated with each other? Here we review findings from a wide range of studies using the auditory streaming paradigm in order to formulate a unified account of the processes underlying auditory perceptual organization. We present new computational modelling results which replicate responses in primary auditory cortex [Fishman, Y.I., Arezzo, J.C., Steinschneider, M., 2004. Auditory stream segregation in monkey auditory cortex: effects of frequency separation, presentation rate, and tone duration. J. Acoust. Soc. Am. 116, 1656-1670; Fishman, Y. I., Reser, D. H., Arezzo, J.C., Steinschneider, M., 2001. Neural correlates of auditory stream segregation in primary auditory cortex of the awake monkey. Hear. Res. 151, 167-187] to tone sequences. We also present the results of a perceptual experiment which confirm the bi-stable nature of auditory streaming, and the proposal that the gradual build-up of streaming may be an artefact of averaging across many subjects [Pressnitzer, D., Hupé, J. M., 2006. Temporal dynamics of auditory and visual bi-stability reveal common principles of perceptual organization. Curr. Biol. 16(13), 1351-1357.]. Finally we argue that in order to account for all of the experimental findings, computational models of auditory stream segregation require four basic processing elements; segregation, predictive modelling, competition and adaptation, and that it is the formation of effective predictive models which allows the system to keep track of different sound sources in a complex auditory environment.

The ‘F-complex’ and MMN tap different aspects of deviance

OBJECTIVE

To compare the 'F(fusion)-complex' with the Mismatch negativity (MMN), both components associated with automatic detection of changes in the acoustic stimulus flow.

METHODS

Ten right-handed adult native Hebrew speakers discriminated vowel-consonant-vowel (V-C-V) sequences /ada/ (deviant) and /aga/ (standard) in an active auditory 'Oddball' task, and the brain potentials associated with performance of the task were recorded from 21 electrodes. Stimuli were generated by fusing the acoustic elements of the V-C-V sequences as follows: base was always presented in front of the subject, and formant transitions were presented to the front, left or right in a virtual reality room. An illusion of a lateralized echo (duplex sensation) accompanied base fusion with the lateralized formant locations. Source current density estimates were derived for the net response to the fusion of the speech elements (F-complex) and for the MMN, using low-resolution electromagnetic tomography (LORETA). Statistical non-parametric mapping was used to estimate the current density differences between the brain sources of the F-complex and the MMN.

RESULTS

Occipito-parietal regions and prefrontal regions were associated with the F-complex in all formant locations, whereas the vicinity of the supratemporal plane was bilaterally associated with the MMN, but only in case of front-fusion (no duplex effect).

CONCLUSIONS

MMN is sensitive to the novelty of the auditory object in relation to other stimuli in a sequence, whereas the F-complex is sensitive to the acoustic features of the auditory object and reflects a process of matching them with target categories.

SIGNIFICANCE

The F-complex and MMN reflect different aspects of auditory processing in a stimulus-rich and changing environment: content analysis of the stimulus and novelty detection, respectively.

Effects of temporal grouping on the memory representation of inter-tone relationships

The length of silence between successive sounds is a dominant cue for temporal grouping of sounds. The present study tested whether the sensory memory representation of inter-tone relationships is dependent on the grouping of tones within a single stream of sound. Subjects were presented with sequences of two alternating tones that differed from each other in frequency. Perception of a sequence made up of tone-pairs was promoted by alternating a short and a long inter-tone interval. Occasional tone repetitions fell either within one tone-pair or across two pairs. We found that detecting tone repetitions was slower for across- than within-pair repetitions (Experiment 1). Also the amplitude of the mismatch negativity (MMN) event-related potential was lower for across-pair repetitions compared with that measured in the control isochronous sequences (Experiment 2). This attenuation of the MMN-amplitude could not be explained by the inter-tone interval differences that existed between the paired and the isochronous conditions (Experiment 3). These results demonstrate that temporal grouping affects the sensory memory representation of inter-tone relationships within a single sound stream.

From Sensory to Long-Term Memory

Abstract. Everyday experience tells us that some types of auditory sensory information are retained for long periods of time. For example, we are able to recognize friends by their voice alone or identify the source of familiar noises even years after we last heard the sounds. It is thus somewhat surprising that the results of most studies of auditory sensory memory show that acoustic details, such as the pitch of a tone, fade from memory in ca. 10-15 s. One should, therefore, ask (1) what types of acoustic information can be retained for a longer term, (2) what circumstances allow or help the formation of durable memory records for acoustic details, and (3) how such memory records can be accessed. The present review discusses the results of experiments that used a model of auditory recognition, the auditory memory reactivation paradigm. Results obtained with this paradigm suggest that the brain stores features of individual sounds embedded within representations of acoustic regularities that have been detected for the sound patterns and sequences in which the sounds appeared. Thus, sounds closely linked with their auditory context are more likely to be remembered. The representations of acoustic regularities are automatically activated by matching sounds, enabling object recognition.

Spatiotemporal characteristics of the neural activities processing consonant/dissonant tones in melody

To identify neural correlates underlying melody processing, we measured MEG responses elicited by keynote and out-of-key tones at the end of musical phrases. These melodies were newly composed and unknown to the subjects. Significant enlargements of N1m/P2m peaks at about 120-160 ms were observed in response to dissonant (out-of-key) tones compared to those in response to consonant (keynote) tones. The equivalent current dipoles (ECD) of the N1m were localized in areas centered at bilateral primary auditory cortices in the superior surface of the temporal lobe. Following the N1m/P2m, a late component occurring at 280-530 ms was observed. As the latency proceeded, the location of ECD sources of the late component shifted in the right hemisphere, but not in the left hemisphere, from the supratemporal auditory cortex to a posterior inferior auditory association cortex around the superior temporal sulcus (STS). The grand mean locations of the ECDs for consonant and dissonant tones were separated at a peak period of 380-410 ms of the late component but converged to the same region around the STS in the last period of 440-530 ms. These observations suggest that the neural activities generating the N1m component in the bilateral auditory cortices may play a role in automatic detection of tonality mismatch based on the pitch analysis. The activities of the late component around the posterior part of the right STS are thought to be involved in the analysis of pitch-sequence, such as how the pitch changes temporally, as a pre-process of melody perception.

The development of children at familial risk for dyslexia: birth to early school age

Children at risk for familial dyslexia (n = 107) and their controls (n = 93) have been followed from birth to school entry in the Jyvaskyla Longitudinal study of Dyslexia (JLD) on developmental factors linked to reading and dyslexia. At the point of school entry, the majority of the at-risk children displayed decoding ability that fell at least 1 SD below the mean of the control group. Measures of speech processing were the earliest indices to show both group differences in infancy and also significant predictive associations with reading acquisition. A number of measures of language, including phonological and morphological skill collected repeatedly from age three, revealed group differences and predictive correlations. Both the group differences and the predictive associations to later language and reading ability strengthened as a function of increasing age. The predictions, however, tend to be stronger and the spectrum of significant correlations wider in the at-risk group. These results are crucial to early identification and intervention of dyslexia in at-risk children.

Sound presentation rate is represented logarithmically in human cortex

The encoding of temporal information is critical to auditory processing. Since the mismatch negativity component of the auditory event-related brain potential is thought to reflect properties of auditory sensory memory, we used it to examine the representation of acoustic time intervals in the human cortex. The mismatch negativity occurs in response to deviations in acoustic regularities, which are stored in sensory memory. We used 16 stimulus conditions, randomly presenting short trains of tones with fixed onset-to-onset intervals of 100, 200, 300 or 400 ms (all tones in the study were identical). The first four intervals between the tones established the acoustic regularity on each train (i.e. the 'standard'). The fifth tone in each train was preceded by an interval that varied randomly among the same four intervals. If this interval was different from the standard for that trial, it violated the acoustic regularity (i.e. it was a 'deviant'). The mismatch response to the fifth tone differed significantly among stimulus conditions and was proportional to the absolute value of the logarithm of the deviant/standard interval ratio. This indicates that short acoustic time intervals are represented with a ratio scale in the human cortex. When the fifth tone occurred 100 ms after the fourth, it elicited a somewhat different, although proportional response, supporting the hypothesis that a special integration mechanism may exist for very short time intervals.

Effects of temporal encoding on auditory object formation: a mismatch negativity study

Analysis of auditory scene relies on the appropriate division of the sensory input into meaningful events. The auditory system uses a sequential grouping strategy to segregate different acoustic events in the time domain. As a result of grouping, the organization of sound is expected to include acoustic events spanning different scales of time. It is unclear, however, at which moment the central auditory system begins to introduce changes in the neural representation of the auditory scene as a result of grouping different information units into a larger acoustic event. The present set of experiments examines this issue by using the mismatch negativity (MMN) component of event-related potentials. The MMN is elicited in response to changes in the previously registered auditory regularity whether or not the subject's attention is focused on the auditory stimuli. By presenting three consecutive pairs of alternating tones in serial stimulus trains, we found that across-train repetitions of the tone elicited the MMN with inter-train intervals up to 240 ms. Beyond this interval, only within-train repetitions of the tone elicited the MMN, suggesting that pairs of tones were bound together into a single acoustic event (i.e., train of six tones) within a temporal window not much longer than 240 ms. Furthermore, a different pattern of responses was seen depending on the position of the deviant event within the train and the frequency distance between the alternating tones. These results suggest that grouping discrete auditory inputs may introduce changes in the neural representation of the auditory scene by applying rules to sounds spanning larger temporal scales.

Brain responses to changes in speech sound durations differ between infants with and without familial risk for dyslexia

A specific learning disability, developmental dyslexia, is a language-based disorder that is shown to be strongly familial. Therefore, infants born to families with a history of the disorder are at an elevated risk for the disorder. However, little is known of the potential early markers of dyslexia. Here we report differences between 6-month-old infants with and without high risk of familial dyslexia in brain electrical activation generated by changes in the temporal structure of speech sounds, a critical cueing feature in speech. We measured event-related brain responses to consonant duration changes embedded in ata pseudowords applying an oddball paradigm, in which pseudoword tokens with varying /t/ duration were presented as frequent standard (80%) or as rare deviant stimuli (each 10%) with an interval of 610 msec between the stimuli. The infants at risk differ from control infants in both their initial responsiveness to sounds per se and in their change-detection responses dependent on the stimulus context. These results show that infants at risk due to a familial background of reading problems process auditory temporal cues of speech sounds differently from infants without such a risk even before they learn to speak.

Preattentive auditory context effects

The effects of auditory context on the preattentive and perceptual organization of tone sequences were investigated. Two sets of experiments were conducted in which the pitch of contextual tones was varied, bringing about two different contextual manipulations. Preattentive auditory organization was indexed by the mismatch negativity event-related potential, which is elicited by violations of auditory regularities even when participants ignore the sounds (e.g., by reading a book). The perceptual effects of the contextual manipulations on auditory grouping were assessed using target-detection and order-judgment tasks. The close correspondence found between the effects of auditory context on the perceptual and preattentive measures of auditory grouping suggests that a large part of contextual processing is preattentive.

Spatial asymmetries of auditory event-synthesis in humans

We used the mismatch negativity event-related potential to examine how spatial location and feature variation affect the capacity of the auditory system to automatically respond to pairs of rapid (180 ms apart) acoustic changes within a single tone. When a tone first deviated from a standard tone in source location and then in its duration, we found independent responses to both deviations for right but not left field stimuli. In contrast, when the first deviation was in pitch and the second in duration, only the first deviation elicited a response, regardless of presentation side. These results suggest that information from either side of space is asymmetrically processed even in a free-field, and that the extent of the temporal window of integration is not a fixed property of the auditory system.

Temporal integration: intentional sound discrimination does not modulate stimulus-driven processes in auditory event synthesis

OBJECTIVE

Our previous study showed that the auditory context could influence whether two successive acoustic changes occurring within the temporal integration window (approximately 200ms) were pre-attentively encoded as a single auditory event or as two discrete events (Cogn Brain Res 12 (2001) 431). The aim of the current study was to assess whether top-down processes could influence the stimulus-driven processes in determining what constitutes an auditory event.

METHODS

Electroencepholagram (EEG) was recorded from 11 scalp electrodes to frequently occurring standard and infrequently occurring deviant sounds. Within the stimulus blocks, deviants either occurred only in pairs (successive feature changes) or both singly and in pairs. Event-related potential indices of change and target detection, the mismatch negativity (MMN) and the N2b component, respectively, were compared with the simultaneously measured performance in discriminating the deviants.

RESULTS

Even though subjects could voluntarily distinguish the two successive auditory feature changes from each other, which was also indicated by the elicitation of the N2b target-detection response, top-down processes did not modify the event organization reflected by the MMN response.

CONCLUSIONS

Top-down processes can extract elemental auditory information from a single integrated acoustic event, but the extraction occurs at a later processing stage than the one whose outcome is indexed by MMN.

SIGNIFICANCE

Initial processes of auditory event-formation are fully governed by the context within which the sounds occur. Perception of the deviants as two separate sound events (the top-down effects) did not change the initial neural representation of the same deviants as one event (indexed by the MMN), without a corresponding change in the stimulus-driven sound organization.

Impairment in frontal but not temporal components of mismatch negativity in schizophrenia

Impairment in mismatch negativity (MMN) potentials is a robust finding in schizophrenia. While previous studies suggested that MMN in man is generated by a single dipole source bilaterally in the primary auditory cortex, more recent data modified this assumption by showing differential modulation of MMN components over the frontal and temporal scalp. Here we used a roving standard experiment to record mismatch potentials to tone duration deviants with the aim to detect robust temporal and frontal mismatch components. Fourteen schizophrenic patients with normal intelligence and without overt cognitive deficits and age- and sex-matched controls were studied. In agreement with previous findings MMN recorded from the frontal scalp was markedly attenuated in patients. However, in contrast to previous reports, positive mismatch potentials of normal magnitude were recorded from temporal (mastoid) electrodes. This finding raises the possibility of a selective impairment in multiple mismatch generators in schizophrenia and may lend support for the notion of impaired cortico-cortical connectivity in schizophrenia.

Dynamic sensory updating in the auditory system

Typically, in everyday situations, auditory input is constantly changing. Change is an important cue for the auditory system, which can signal the start of new sources of information or that some action may be required. Using an event-related brain potential that can be elicited whether or not attention is focused on the sounds (the mismatch negativity, MMN) we measured the time course of the effects of contextual changes on the brain's response to the same stimulus event. The onset or cessation of a sound in a stimulus block brought about context changes. The effect of the context was observed through changes in the MMN response to a deviant event that was present throughout the sound sequence. These results suggest the existence of a dynamic system of change detection, which updates its model of the sensory input on-line as the changes occur.

Simultaneously active pre-attentive representations of local and global rules for sound sequences in the human brain

Regular sequences of sounds (i.e., non-random) can usually be described by several, equally valid rules. Rules allowing extrapolation from one sound to the next are termed local rules, those that define relations between temporally non-adjacent sounds are termed global rules. The aim of the present study was to determine whether both local and global rules can be simultaneously extracted from a sound sequence even when attention is directed away from the auditory stimuli. The pre-attentive representation of a sequence of two alternating tones (differing only in frequency) was investigated using the mismatch negativity (MMN) auditory event-related potential. Both local- and global-rule violations of tone alternation elicited the MMN component while subjects ignored the auditory stimuli. This finding suggests that (a) pre-attentive auditory processes can extract both local and global rules from sound sequences, and (b) that several regularity representations of a sound sequence are simultaneously maintained during the pre-attentive phase of auditory stimulus processing.

"Primitive intelligence" in the auditory cortex

The everyday auditory environment consists of multiple simultaneously active sources with overlapping temporal and spectral acoustic properties. Despite the seemingly chaotic composite signal impinging on our ears, the resulting perception is of an orderly "auditory scene" that is organized according to sources and auditory events, allowing us to select messages easily, recognize familiar sound patterns, and distinguish deviant or novel ones. Recent data suggest that these perceptual achievements are mainly based on processes of a cognitive nature ("sensory intelligence") in the auditory cortex. Even higher cognitive processes than previously thought, such as those that organize the auditory input, extract the common invariant patterns shared by a number of acoustically varying sounds, or anticipate the auditory events of the immediate future, occur at the level of sensory cortex (even when attention is not directed towards the sensory input).

Do perceived loudness cues contribute to duration mismatch negativity (MMN)?

This study explored the contribution of perceived loudness cues to mismatch negativity produced in response to a 125 ms duration deviant tone among a regular sequence of 50 ms standard tones. Each individual was required to adjust the intensity of a 125 ms tone until it matched the perceived loudness of a 50 ms tone. The mismatch negativity produced to the duration deviant presented at the same intensity as the standard was contrasted with that produced to the same deviant at each individual's perceived loudness equivalence. Despite detectable difference perceived loudness (approximately 1.3 dB), adjusting the intensity of the deviant tone did not significantly reduce mismatch negativity amplitude to the duration deviant. Results are discussed with reference to temporal window of integration.

Associated deficits in mismatch negativity generation and tone matching in schizophrenia

OBJECTIVE

Schizophrenia is associated with deficits in mismatch negativity (MMN) generation and in the ability to match two tones following brief delay. Both deficits reflect impaired early cortical processing of auditory information. However, the relationship between deficits in MMN generation and tone matching performance in schizophrenia has not been established.

METHODS

MMN and tone matching performance was evaluated in 12 schizophrenic subjects and 12 similar aged controls. A pitch separation known to produce non-ceiling performance in patients (5% Deltaf) was used. Narrow band filtering of MMN data was used to enhance signal-to-noise ratio.

RESULTS

Schizophrenic subjects showed impairments in both MMN generation and tone matching performance. The two deficits were significantly correlated across subjects. In addition, decreased MMN amplitude and poorer tone matching performance correlated with increased severity of negative symptoms.

CONCLUSIONS

These findings support the concept that similar neurophysiological mechanisms underlie MMN and tone matching deficits in schizophrenia. Further, they indicate that increased sensitivity to environmental change may be related to social withdrawal and other negative symptoms in schizophrenia.

Temporal integration of auditory stimulus deviance as reflected by the mismatch negativity

We recorded event-related brain potentials (ERPs) to two different infrequent deviant tones presented successively within the repetitive sequence of a standard tone. A separate mismatch negativity (MMN) component was elicited by each of the two deviants when the interval separating their onsets was 300 ms. However, only a single MMN component was elicited when the temporal separation between the onsets of the two deviants was 150 ms. Previous studies obtained similar results using two temporally separated deviations carried by a single sound. Taken together, these results support the notion of a general temporal integration mechanism in the formation of auditory events with ca. 200 ms long window.