Phonetic invariance in the human auditory cortex

Gradient boosted decision trees reveal nuances of auditory discrimination behavior

Animal psychophysics can generate rich behavioral datasets, often comprised of many 1000s of trials for an individual subject. Gradient-boosted models are a promising machine learning approach for analyzing such data, partly due to the tools that allow users to gain insight into how the model makes predictions. We trained ferrets to report a target word's presence, timing, and lateralization within a stream of consecutively presented non-target words. To assess the animals' ability to generalize across pitch, we manipulated the fundamental frequency (F0) of the speech stimuli across trials, and to assess the contribution of pitch to streaming, we roved the F0 from word token to token. We then implemented gradient-boosted regression and decision trees on the trial outcome and reaction time data to understand the behavioral factors behind the ferrets' decision-making. We visualized model contributions by implementing SHAPs feature importance and partial dependency plots. While ferrets could accurately perform the task across all pitch-shifted conditions, our models reveal subtle effects of shifting F0 on performance, with within-trial pitch shifting elevating false alarms and extending reaction times. Our models identified a subset of non-target words that animals commonly false alarmed to. Follow-up analysis demonstrated that the spectrotemporal similarity of target and non-target words rather than similarity in duration or amplitude waveform was the strongest predictor of the likelihood of false alarming. Finally, we compared the results with those obtained with traditional mixed effects models, revealing equivalent or better performance for the gradient-boosted models over these approaches.

Unified Coding of Spectral and Temporal Phonetic Cues: Electrophysiological Evidence for Abstract Phonological Features

Spoken word recognition models and phonological theory propose that abstract features play a central role in speech processing. It remains unknown, however, whether auditory cortex encodes linguistic features in a manner beyond the phonetic properties of the speech sounds themselves. We took advantage of the fact that English phonology functionally codes stops and fricatives as voiced or voiceless with two distinct phonetic cues: Fricatives use a spectral cue, whereas stops use a temporal cue. Evidence that these cues can be grouped together would indicate the disjunctive coding of distinct phonetic cues into a functionally defined abstract phonological feature. In English, the voicing feature, which distinguishes the consonants [s] and [t] from [z] and [d], respectively, is hypothesized to be specified only for voiceless consonants (e.g., [s t]). Here, participants listened to syllables in a many-to-one oddball design, while their EEG was recorded. In one block, both voiceless stops and fricatives were the standards. In the other block, both voiced stops and fricatives were the standards. A critical design element was the presence of intercategory variation within the standards. Therefore, a many-to-one relationship, which is necessary to elicit an MMN, existed only if the stop and fricative standards were grouped together. In addition to the ERPs, event-related spectral power was also analyzed. Results showed an MMN effect in the voiceless standards block—an asymmetric MMN—in a time window consistent with processing in auditory cortex, as well as increased prestimulus beta-band oscillatory power to voiceless standards. These findings suggest that (i) there is an auditory memory trace of the standards based on the shared (voiceless) feature, which is only functionally defined; (ii) voiced consonants are underspecified; and (iii) features can serve as a basis for predictive processing. Taken together, these results point toward auditory cortex's ability to functionally code distinct phonetic cues together and suggest that abstract features can be used to parse the continuous acoustic signal.

Do categorical representations modulate early automatic visual processing? A visual mismatch-negativity study

Perceptual categorization is an important cognitive function. In the auditory domain, categorization already occurs within the first 200 ms of information processing, as indexed by the mismatch negativity. Here, we assessed the characteristics of the visual mismatch negativity (vMMN) elicited during the categorization of previously unknown visual stimuli. To examine this, we used five-dot patterns with characteristics that allow for the formation of categories through rotation and reflection but not through other physical properties. To assess whether or not between-category and within-category vMMN differ in amplitude, the data was analyzed with the Bayesian approach. We observed that both between-category and within-category deviants elicited a vMMN, but that both vMMNs were comparable in magnitude. This finding suggests that abstract categorical representations are not always automatically processed at early visual stages and demonstrates limitations of generalization from the auditory domain to visual domain.

Extracting Phonetic Features From Natural Classes: A Mismatch Negativity Study of Mandarin Chinese Retroflex Consonants

How speech sounds are represented in the brain is not fully understood. The mismatch negativity (MMN) has proven to be a powerful tool in this regard. The MMN event-related potential is elicited by a deviant stimulus embedded within a series of repeating standard stimuli. Listeners construct auditory memory representations of these standards despite acoustic variability. In most designs that test speech sounds, however, this variation is typically intra-category: All standards belong to the same phonetic category. In the current paper, inter-category variation is presented in the standards. These standards vary in manner of articulation but share a common phonetic feature. In the standard retroflex experimental block, Mandarin Chinese speaking participants are presented with a series of "standard" consonants that share the feature [retroflex], interrupted by infrequent non-retroflex deviants. In the non-retroflex standard experimental block, non-retroflex standards are interrupted by infrequent retroflex deviants. The within-block MMN was calculated, as was the identity MMN (iMMN) to account for intrinsic differences in responses to the stimuli. We only observed a within-block MMN to the non-retroflex deviant embedded in the standard retroflex block. This suggests that listeners extract [retroflex] despite significant inter-category variation. In the non-retroflex standard block, because there is little on which to base a coherent auditory memory representation, no within-block MMN was observed. The iMMN to the retroflex was observed in a late time-window at centro-parieto-occipital electrode sites instead of fronto-central electrodes, where the MMN is typically observed, potentially reflecting the increased difficulty posed by the added variation in the standards. In short, participants can construct auditory memory representations despite significant acoustic and inter-category phonological variation so long as a shared phonetic feature binds them together.

Using rotated speech to approximate the acoustic mismatch negativity response to speech

The mismatch negativity (MMN) response is influenced by the magnitude of the acoustic difference between standard and deviant, and the response is typically larger to linguistically relevant changes than to linguistically irrelevant changes. Linguistically relevant changes between standard and deviant typically co-occur with differences between the two acoustic signals. It is therefore not straightforward to determine the contribution of each of those two factors to the MMN response. This study investigated whether spectrally rotated speech can be used to determine the impact of the acoustic difference on the MMN response to a combined linguistic and acoustic change between standard and deviant. Changes between rotated vowels elicited an MMN of comparable amplitude to the one elicited by a within-category vowel change, whereas the between-category vowel change resulted in an MMN amplitude of greater magnitude. A change between rotated vowels resulted in an MMN ampltude more similar to that of a within-vowel change than a complex tone change did. This suggests that the MMN amplitude reflecting the acoustic difference between two speech sounds can be well approximated by the MMN amplitude elicited in response to their rotated counterparts, in turn making it possible to estimate the part of the response specific to the linguistic difference.

N1 response attenuation and the mismatch negativity (MMN) to within- and across-category phonetic contrasts

According to the neural adaptation model of the mismatch negativity (MMN), the sensitivity of this event-related response to both acoustic and categorical information in speech sounds can be accounted for by assuming that (a) the degree of overlapping between neural representations of two sounds depends on both the acoustic difference between them and whether or not they belong to distinct phonetic categories, and (b) a release from stimulus-specific adaptation causes an enhanced N1 obligatory response to infrequent deviant stimuli. On the basis of this view, we tested in Experiment 1 whether the N1 response to the second sound of a pair (S₂ ) would be more attenuated in pairs of identical vowels compared with pairs of different vowels, and in pairs of exemplars of the same vowel category compared with pairs of exemplars of different categories. The psychoacoustic distance between S₁ and S₂ was the same for all within-category and across-category pairs. While N1 amplitudes decreased markedly from S₁ to S₂ , responses to S₂ were quite similar across pair types, indicating that the attenuation effect in such conditions is not stimulus specific. In Experiment 2, a pronounced MMN was elicited by a deviant vowel sound in an across-category oddball sequence, but not when the exact same deviant vowel was presented in a within-category oddball sequence. This adds evidence that MMN reflects categorical phonetic processing. Taken together, the results suggest that different neural processes underlie the attenuation of the N1 response to S₂ and the MMN to vowels.

The Processing of Biologically Plausible and Implausible forms in American Sign Language: Evidence for Perceptual Tuning

The human auditory system distinguishes speech-like information from general auditory signals in a remarkably fast and efficient way. Combining psychophysics and neurophysiology (MEG), we demonstrate a similar result for the processing of visual information used for language communication in users of sign languages. We demonstrate that the earliest visual cortical responses in deaf signers viewing American Sign Language (ASL) signs show specific modulations to violations of anatomic constraints that would make the sign either possible or impossible to articulate. These neural data are accompanied with a significantly increased perceptual sensitivity to the anatomical incongruity. The differential effects in the early visual evoked potentials arguably reflect an expectation-driven assessment of somatic representational integrity, suggesting that language experience and/or auditory deprivation may shape the neuronal mechanisms underlying the analysis of complex human form. The data demonstrate that the perceptual tuning that underlies the discrimination of language and non-language information is not limited to spoken languages but extends to languages expressed in the visual modality.

Behavioral correlates of auditory streaming in rhesus macaques

Perceptual representations of auditory stimuli (i.e., sounds) are derived from the auditory system's ability to segregate and group the spectral, temporal, and spatial features of auditory stimuli-a process called "auditory scene analysis". Psychophysical studies have identified several of the principles and mechanisms that underlie a listener's ability to segregate and group acoustic stimuli. One important psychophysical task that has illuminated many of these principles and mechanisms is the "streaming" task. Despite the wide use of this task to study psychophysical mechanisms of human audition, no studies have explicitly tested the streaming abilities of non-human animals using the standard methodologies employed in human-audition studies. Here, we trained rhesus macaques to participate in the streaming task using methodologies and controls similar to those presented in previous human studies. Overall, we found that the monkeys' behavioral reports were qualitatively consistent with those of human listeners, thus suggesting that this task may be a valuable tool for future neurophysiological studies.

A sparse neural code for some speech sounds but not for others

The precise neural mechanisms underlying speech sound representations are still a matter of debate. Proponents of 'sparse representations' assume that on the level of speech sounds, only contrastive or otherwise not predictable information is stored in long-term memory. Here, in a passive oddball paradigm, we challenge the neural foundations of such a 'sparse' representation; we use words that differ only in their penultimate consonant ("coronal" [t] vs. "dorsal" [k] place of articulation) and for example distinguish between the German nouns Latz ([lats]; bib) and Lachs ([laks]; salmon). Changes from standard [t] to deviant [k] and vice versa elicited a discernible Mismatch Negativity (MMN) response. Crucially, however, the MMN for the deviant [lats] was stronger than the MMN for the deviant [laks]. Source localization showed this difference to be due to enhanced brain activity in right superior temporal cortex. These findings reflect a difference in phonological 'sparsity': Coronal [t] segments, but not dorsal [k] segments, are based on more sparse representations and elicit less specific neural predictions; sensory deviations from this prediction are more readily 'tolerated' and accordingly trigger weaker MMNs. The results foster the neurocomputational reality of 'representationally sparse' models of speech perception that are compatible with more general predictive mechanisms in auditory perception.

Mismatch negativity for speech sounds in temporal lobe epilepsy

The mismatch negativity (MMN) is an electrophysiological trace of change detection, measured by electroencephalography (EEG), and is a reliable marker for pre-attentive auditory sensory memory. We used a phonetic oddball paradigm in patients with temporal lobe epilepsy (TLE) to elicit the MMN response at fronto-central sites and the mismatch positivity (MMP) response at mastoid sites. The MMN in 26 patients was compared with that of 26 age- and gender-matched healthy control participants. Electroencephalography responses were recorded during the presentation of speech sounds: the vowels 'a' and 'o' in alternation. Average waveforms were obtained for standard and deviant trials. We found that the MMP response at bilateral mastoid sites was reduced, whereas the MMN response at fronto-central sites did not change significantly. These results support the view that the MMN is generated by separable sources in the frontal and temporal lobes and that these sources are differentially affected by TLE.

The mismatch negativity (MMN) in basic research of central auditory processing: a review

In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.

The perception of invariant speech features in children with autism

We investigated whether the good pitch-discrimination abilities reported in individuals with autism have adverse effects on their speech perception by compromising their ability to extract invariant phonetic features from speech input. The MMN, a brain response reflecting sound-discrimination processes, was recorded from children with autism and their controls for phoneme-category and pitch changes in speech stimuli under two different conditions: (a) when all the other features of the standard and deviant stimuli were kept constant, and (b) when constant variation with respect to an irrelevant feature was introduced to the standard and deviant stimuli. Children with autism had enhanced MMNs for pitch changes in both conditions, as well as for phoneme-category changes in the constant-feature condition. However, when the phoneme-category changes occurred in phonemes having pitch variation, the MMN enhancement was abolished in autistic children. This suggests that children with autism lose their advantage in phoneme discrimination when the context of the stimuli is speech-like and requires abstracting invariant speech features from varying input.

Social cognition and epilepsy surgery

Human social behavior depends on a set of perceptive, mnemonic, and interpretive abilities that together may be termed social cognition. Lesion and functional imaging studies of social cognitive functions implicate the temporal lobes (in particular, the nondominant temporal lobe) and mesial temporal structures as critical at the front end of social cognitive processes. The frontal lobes, in turn, function to interpret and to modulate these processes via top-down control. Damage to frontal regions is associated with specific derangements in social behavior. Chronic focal-onset epilepsy and its surgical treatment commonly affect these neuroanatomic regions and might therefore impact social function. Postoperative social function helps determine quality of life for both patients and families. There is some evidence that resective seizure surgery affects social cognition, but there are significant weaknesses in our current knowledge that can be overcome with comprehensive longitudinal research.

Input to verbal working memory: Preattentive construction of the central speech representation

Working memory uses central sound representations as an informational basis. The central sound representation is the temporally and feature-integrated mental representation that corresponds to phenomenal perception. It is used in (higher-order) mental operations and stored in long-term memory. In the bottom-up processing path, the central sound representation can be probed at the level of auditory sensory memory with the mismatch negativity (MMN) of the event-related potential. The present paper reviews a newly developed MMN paradigm to tap into the processing of speech sound representations. Preattentive vowel categorization based on F1-F2 formant information occurs in speech sounds and complex tones even under conditions of high variability of the auditory input. However, an additional experiment demonstrated the limits of the preattentive categorization of language-relevant information. It tested whether the system categorizes complex tones containing the F1 and F2 formant components of the vowel /a/ differently than six sounds with nonlanguage-like F1-F2 combinations. From the absence of an MMN in this experiment, it is concluded that no adequate vowel representation was constructed. This shows limitations of the capability of preattentive vowel categorization.

The effect of 1/f fluctuation in inter-stimulus intervals on auditory evoked mismatch field

This study focused on the effect of regularity of environmental stimuli on the informational order extracting function of human brain. The regularity of environmental stimuli can be described with the exponent n of the fluctuation 1/f(n). We studied the effect of the exponent of the fluctuation in the inter-stimulus interval (ISI) on the elicitation of auditory evoked mismatch fields (MMF) with two sounds with alternating frequency. ISI times were given by three types of fluctuation, 1/f(0), 1/f(1), 1/f(2), and with a fixed interval (1/f(infinity)). The root mean square (RMS) value of the MMF increased significantly (F(3/9)=4.95, p=0.027) with increases in the exponent of the fluctuation. Increments in the regularity of the fluctuation provoked enhancement of the MMF, which reflected the production of a memory trace, based on the anticipation of the stimulus timing. The gradient of the curve, indicating the ratio of increments between the MMF and the exponent of fluctuation, can express a subject's capability to extract regularity from fluctuating stimuli.

Cortical differentiation of speech and nonspeech sounds at 100 ms: implications for dyslexia

Neurophysiological measures indicate cortical sensitivity to speech sounds by 150 ms after stimulus onset. In this time window dyslexic subjects start to show abnormal cortical processing. We investigated whether phonetic analysis is reflected in the robust auditory cortical activation at approximately 100 ms (N100m), and whether dyslexic subjects show abnormal N100m responses to speech or nonspeech sounds. We used magnetoencephalography to record auditory responses of 10 normally reading and 10 dyslexic adults. The speech stimuli were synthetic Finnish speech sounds (/a/, /u/, /pa/, /ka/). The nonspeech stimuli were complex nonspeech sounds and simple sine wave tones, composed of the F1+F2+F3 and F2 formant frequencies of the speech sounds, respectively. All sounds evoked a prominent N100m response in the bilateral auditory cortices. The N100m activation was stronger to speech than nonspeech sounds in the left but not in the right auditory cortex, in both subject groups. The leftward shift of hemispheric balance for speech sounds is likely to reflect analysis at the phonetic level. In dyslexic subjects the overall interhemispheric amplitude balance and timing were altered for all sound types alike. Dyslexic individuals thus seem to have an unusual cortical organization of general auditory processing in the time window of speech-sensitive analysis.

Pre-attentive categorization of vowel formant structure in complex tones

It has been demonstrated that vowel information can be extracted from speech sounds without attention focused on them, despite widely varying non-speech acoustic information in the input. The present study tested whether even complex tones that were constructed based on F0, F1 and F2 vowel frequencies to resemble the defining features of speech sounds, but were not speech, are categorized pre-attentively according to vowel space information. The Mismatch Negativity brain response was elicited by infrequent tokens of the complex tones, showing that the auditory system can pre-attentively categorize speech information on the basis of the minimal, defining auditory features. The human mind extracts the language-relevant information from complex tones despite the non-relevant variation in the sound input.

Pre-attentive perception of vowel phonemes from variable speech stimuli

Understanding speech requires the construction of phonetic representations while abstracting from specific sound features. To understand different speakers of varying pitches of voice, loudness, or timbre, categorical phoneme information needs to be rapidly extracted from dynamic, changing speech input. The present study demonstrated a genuine MMN to tokens of /a/ and /i/ vowels varying in pitch of voice and amplitude envelope when they occurred infrequently among the respective other vowels. These data indicate that the speech perception system pre-attentively extracted the F1/F2 formant information despite the language-irrelevant variation in the sound input.

Automatic Auditory Processing of English Words as Indexed by the Mismatch Negativity, Using a Multiple Deviant Paradigm

OBJECTIVE

The aim of this study was to investigate mismatch negativity (MMN) responses to a variety of speech stimuli (/de:/, /ge:/, /deI/ "day", and /geI/ "gay") in a multiple deviant paradigm. It was hypothesized that all speech stimulus contrasts in the multiple deviant paradigm, including the fine acoustic speech contrast [d/g], would elicit robust MMN responses and that consonant vowel (CV) real word deviants (e.g., "day" and "gay") would elicit larger MMN responses than CV nonword deviants (e.g., "de" and "ge") within and across experimental contrasts.

DESIGN

Ten healthy, right-handed, native English-speaking adults (23.4 +/- 2.27 yr) with normal hearing were presented with 12 blocks of stimuli, using a multiple deviant oddball paradigm. Each of the four speech stimuli were presented as standards (p = 0.7) in three blocks, with the remaining stimuli acting as deviants (p = 0.1 each). Subjects were also presented with the same stimuli in a behavioral discrimination task.

RESULTS

MMN responses to the fine acoustic speech contrast [d/g] (e.g., "de" versus "ge", "day" versus "gay") did not reach significance. However, a significant and larger MMN response was obtained at an earlier latency to the real word deviants among nonword standards with the same initial consonant (i.e., de-->day, ge-->gay) when compared with the responses to nonword deviants among word standards (day-->de, gay-->ge).

CONCLUSIONS

The results showed that MMN responses could be elicited by speech stimuli with large, single acoustic deviances, within a multiple deviant paradigm design. This result has positive clinical implications for the testing of subjects who may only tolerate short testing sessions (e.g., pathological populations) in that responses to a wider range of speech stimuli may be recorded without necessarily having to increase session length. The results also demonstrated that MMN responses were elicited by large, single acoustic deviances but not fine acoustic deviances within the speech stimuli. The poor results for the fine acoustic deviances support previous studies that have used single contrast paradigms and found that when carefully controlled methodological designs and strict methods of analysis are applied, robust responses to fine-grained CV syllable contrasts may be difficult to obtain. The enhanced MMN observed in response to the real word deviants among nonword standards may provide further evidence for the presence of long-term neural traces for words in the brain, however possible contextual effects limit the interpretation of these data. Further research is needed to investigate the ability of the MMN response to accurately reflect speech sounds with fine acoustic contrasts, as well as the ability of the MMN to reflect neural traces for words in the brain, before it can be reliably used as a clinical tool in the investigation of spoken word processing in pathological populations.

Spectral and temporal stimulus characteristics in the processing of abstract auditory features

The processing of abstract stimulus features in the human brain was studied by presenting the subjects with frequent standard tone pairs and infrequent deviant tone pairs. Both pairs varied randomly over a wide frequency and/or intensity range, there being no physically constant standard stimulus. The common feature of the standard pairs was the direction of change within the pair, e.g. the second tone was louder in intensity and/or higher in frequency than the first tone. Deviant pairs, having opposite feature-change direction, elicited the mismatch-negativity (MMN) event-related potential component. MMN was similar to deviations in the direction of frequency and intensity changes and showed no additivity for simultaneous changes in both feature directions. Moreover, MMN was elicited even when the within-pair interval exceeded the 200 ms limit of auditory temporal integration. Results demonstrate that extraction of abstract features is not limited to frequency-based rules, nor is it dependent on temporal integration mechanisms. The lack of MMN additivity between violations of multiple abstract rules suggests that the processing of higher-order invariances differs from that of simple physical features.

Cortical evidence of the perceptual backward masking effect on /l/ and /r/ sounds from a following vowel in Japanese speakers

We examined the influence of stimulus duration of foreign consonant vowel stimuli on the MMNm (magnetic counter part of mismatch negativity). In Experiment 1, /ra/ and /la/ stimuli were synthesized and subjects were native Japanese speakers who are known to have difficulty discriminating the stimuli. "Short" duration stimuli were terminated in the middle of the consonant-to-vowel transition (110 ms). They were nevertheless clearly identifiable by English speakers. A clear MMNm was observed only for short-duration stimuli but not for untruncated long-duration (150-ms) stimuli. We suggest that the diminished MMNm for longer duration stimuli result from more effective masking by the longer vowel part. In Experiment 2 we examined this hypothesis by presenting only the third formant (F3) component of the original stimuli, since the acoustic difference between /la/ and /ra/ is most evident in the third formant, whereas F1 and F2 play a major role in vowel perception. If the MMNm effect depends on the acoustic property of F3, a stimulus duration effect comparable to that found with the original /la/ and /ra/ stimuli might be expected. However, if the effect is attributable to the masking effect from the vowel, no influence of stimulus duration would be expected, since neither stimulus contains F1 and F2 components. In fact, the results showed that the "F3 only" stimuli did not show a duration effect; MMNm was always elicited independent of stimulus duration. The MMN stimulus duration effect is thus suggested to come from the backward masking of foreign consonants by subsequent vowels.

Abstract phoneme representations in the left temporal cortex: magnetic mismatch negativity study

We investigated the brain mechanisms enabling one automatically discriminate phoneme category irrespective of the large inter-speaker variability in the acoustic features of the voices. For this purpose, subjects were presented with 450 different speech stimuli, each uttered by a different speaker, belonging to three vowel categories, while a 306-channel magnetoencephalogram (MEG) was obtained to record the magnetic counterpart of the mismatch negativity (MMNm), elicited only when sensory memory traces for repetitive sounds are formed in the auditory cortex. Despite this wide acoustic variation, category changes elicited prominent MMNm responses, which were considerably stronger in the left than in the right hemisphere in the right-handed subjects. These results implicate the presence of long-term memory traces for vowels, which can recognize the vowel-specific invariant code enabling correct vowel percept even in the presence of realistic acoustic variation.

Cortical processing of change detection: dissociation between natural vowels and two-frequency complex tones

We compared magnetoencephalographic responses for natural vowels and for sounds consisting of two pure tones that represent the two lowest formant frequencies of these vowels. Our aim was to determine whether spectral changes in successive stimuli are detected differently for speech and nonspeech sounds. The stimuli were presented in four blocks applying an oddball paradigm (20% deviants, 80% standards): (i) /alpha/ tokens as deviants vs. /i/ tokens as standards; (ii) /e/ vs. /i/; (iii) complex tones representing /alpha/ formants vs. /i/ formants; and (iv) complex tones representing /e/ formants vs. /i/ formants. Mismatch fields (MMFs) were calculated by subtracting the source waveform produced by standards from that produced by deviants. As expected, MMF amplitudes for the complex tones reflected acoustic deviation: the amplitudes were stronger for the complex tones representing /alpha/ than /e/ formants, i.e., when the spectral difference between standards and deviants was larger. In contrast, MMF amplitudes for the vowels were similar despite their different spectral composition, whereas the MMF onset time was longer for /e/ than for /alpha/. Thus the degree of spectral difference between standards and deviants was reflected by the MMF amplitude for the nonspeech sounds and by the MMF latency for the vowels.

Speaking modifies voice-evoked activity in the human auditory cortex

The voice we most often hear is our own, and proper interaction between speaking and hearing is essential for both acquisition and performance of spoken language. Disturbed audiovocal interactions have been implicated in aphasia, stuttering, and schizophrenic voice hallucinations, but paradigms for a noninvasive assessment of auditory self-monitoring of speaking and its possible dysfunctions are rare. Using magnetoencephalograpy we show here that self-uttered syllables transiently activate the speaker's auditory cortex around 100 ms after voice onset. These phasic responses were delayed by 11 ms in the speech-dominant left hemisphere relative to the right, whereas during listening to a replay of the same utterances the response latencies were symmetric. Moreover, the auditory cortices did not react to rare vowel changes interspersed randomly within a series of repetitively spoken vowels, in contrast to regular change-related responses evoked 100-200 ms after replayed rare vowels. Thus, speaking primes the human auditory cortex at a millisecond time scale, dampening and delaying reactions to self-produced "expected" sounds, more prominently in the speech-dominant hemisphere. Such motor-to-sensory priming of early auditory cortex responses during voicing constitutes one element of speech self-monitoring that could be compromised in central speech disorders.

Neuronal populations in the human brain extracting invariant relationships from acoustic variance

The ability to extract invariant relationships from physically varying stimulation is critical for example to categorical perception of complex auditory information such as speech and music. Human subjects were presented with tone pairs randomly varying over a wide frequency range, there being no physically constant tone pair at all. Instead, the invariant feature was either the direction of the tone pairs (ascending: the second tone was higher in frequency than the first tone) or the frequency ratio (musical interval) of the two tones. The subjects ignored the tone pairs, and instead attended a silent video. Occasional deviant pairs (either descending in direction or having a different frequency ratio) elicited the mismatch negativity (MMN) of the event-related potential, demonstrating the existence of neuronal populations which automatically (independently of attention) extract invariant relationships from acoustical variance.

Pre-attentive detection of vowel contrasts utilizes both phonetic and auditory memory representations

Event-related brain potentials (ERP) were recorded to infrequent changes of a synthesized vowel (standard) to another vowel (deviant) in speakers of Hungarian and Finnish language, which are remotely related to each other with rather similar vowel systems. Both language groups were presented with identical stimuli. One standard-deviant pair represented an across-vowel category contrast in Hungarian, but a within-category contrast in Finnish, with the other pair having the reversed role in the two languages. Both within- and across-category contrasts elicited the mismatch negativity (MMN) ERP component in the native speakers of either language. The MMN amplitude was larger in across- than within-category contrasts in both language groups. These results suggest that the pre-attentive change-detection process generating the MMN utilized both auditory (sensory) and phonetic (categorical) representations of the test vowels.

Hemispheric lateralization in preattentive processing of speech sounds

Event-related magnetoencephalographic (MEG) responses to infrequently presented spoken deviant syllables [di] and [ba] among repetitive standard [da)]syllables were recorded in subjects who either attended to these stimuli in order to discriminate the [ba] syllables or ignored them while attending a silent movie. In both conditions, the deviant syllables elicited a mismatch response (MMNm, the magnetic counterpart of mismatch negativity), which was stronger in the left than in the right auditory cortex, indicating left-hemispheric dominance in speech processing already at a preattentive processing level.

Reactivation of a dormant representation of an auditory stimulus feature

Research with the mismatch negativity component of event-related potentials has uncovered a system that detects change in the acoustic environment on an automatic basis. The system is considered to compare incoming stimuli to representations of the past and to emit an MMN if change is detected. Previous investigations have shown that the relevant memory of the past can become dormant and then be reactivated by a reminder stimulus. It is unclear, however, whether what is reactivated is an holistic representation of stimuli or separate representations of features of stimuli. The present study provides data that supports the latter possibility but leaves open the former one.

Feature conjunctions and auditory sensory memory

This study sought to obtain additional evidence that transient auditory memory stores information about conjunctions of features on an automatic basis. The mismatch negativity of event-related potentials was employed because its operations are based on information that is stored in transient auditory memory. The mismatch negativity was found to be elicited by a tone that differed from standard tones in a combination of its perceived location and frequency. The result lends further support to the hypothesis that the system upon which the mismatch negativity relies processes stimuli in an holistic manner.

Differential effects of alcohol on the cortical processing of foreign and native language

The effect of alcohol (ethanol) on cortical processing of Finnish vs. English words in Finnish-speaking subjects was studied by recording auditory event-related potentials in 10 subjects who had started studying English at the age of 9-10 years. At the beginning of the block of 100 words, the subject heard an introductory sentence. Half of the words completed the sentence well and the other half did not. The subject pressed a reaction key immediately after hearing a proper word. After the control condition, the subject ingested alcohol (1 ml/kg). Alcohol attenuated the amplitude of N100 to both Finnish and English words, this attenuation being significantly stronger for English than for Finnish words. The early differential effect of alcohol suggests that language-specific information is extracted in the cortex already approximately 100 ms from the word onset. The results are in line with animal experiments demonstrating that alcohol selectively affects the activity of single units involved in newer forms of behavior.

Language-specific phoneme representations revealed by electric and magnetic brain responses

There is considerable debate about whether the early processing of sounds depends on whether they form part of speech. Proponents of such speech specificity postulate the existence of language-dependent memory traces, which are activated in the processing of speech but not when equally complex, acoustic non-speech stimuli are processed. Here we report the existence of these traces in the human brain. We presented to Finnish subjects the Finnish phoneme prototype /e/ as the frequent stimulus, and other Finnish phoneme prototypes or a non-prototype (the Estonian prototype /õ/) as the infrequent stimulus. We found that the brain's automatic change-detection response, reflected electrically as the mismatch negativity (MMN), was enhanced when the infrequent, deviant stimulus was a prototype (the Finnish /ö/) relative to when it was a non-prototype (the Estonian /õ/). These phonemic traces, revealed by MMN, are language-specific, as /õ/ caused enhancement of MMN in Estonians. Whole-head magnetic recordings located the source of this native-language, phoneme-related response enhancement, and thus the language-specific memory traces, in the auditory cortex of the left hemisphere.

Processing of vowels in supratemporal auditory cortex

The auditory evoked neuromagnetic fields elicited by synthesized vowels of two different fundamental frequencies F0 were recorded in six subjects over the left and right temporal cortices using a 37-channel biomagnetometer. Single equivalent current dipole modeling of the fields elicited by all vowel types localized activity to a well-circumscribed area in supratemporal auditory cortex in both hemispheres. There were hemisphere asymmetries in the amplitude and latency of the M100 response. We also observed changes in M100 latency related to vowel type, but not to F0. There was no clear effect of vowel type or F0 on dipole localization for the M100, but a possible vowel type by latency interaction. These M100 data provide further evidence that vowels are processed independently of their pitch.

Generators of electrical and magnetic mismatch responses in humans

Studies bearing on generators of the electric and magnetic mismatch responses in humans to change in a repetitive sound are reviewed. It was concluded that the main contribution of the mismatch negativity (MMN) and of its magnetic equivalent MMNm, elicited even in the absence of attention, originates from auditory cortex on the supratemporal plane. In addition, those responses probably have one or two sources on the right lateral temporal cortex and, at least the MMN, a source also in the right frontal cortex. The activation caused by stimulus change in the sensory-specific cortex is, presumably, a manifestation of preperceptual change detection, whereas the frontal activation might be associated with conscious perception of, attention switch to, stimulus change.

Mismatch negativity indicates vowel discrimination in newborns

The present study shows that an infrequent vowel ('deviant') presented among frequent vowels ('standard') elicits in sleeping human newborns a negativity in the auditory event-related potential (ERP) resembling the mismatch negativity (MMN) recorded in adults. Thus, the MMN appears to provide means to investigate brain mechanisms of vowel perception in infants.

Magnetoencephalography in studies of human cognitive brain function

Magnetoencephalography provides a new dimension to the functional imaging of the brain. The cerebral magnetic fields recorded noninvasively enable the accurate determination of locations of cerebral activity with an uncompromized time resolution. The first whole-scalp sensor arrays have just recently come into operation, and significant advances are to be expected in both neurophysiological and cognitive studies, as well as in clinical practice. However, although the accuracy of locating isolated sources of brain activity has improved, identification of multiple simultaneous sources can still be a problem. Therefore, attempts are being made to combine magnetoencephalography with other brain-imaging methods to improve spatial localization of multiple sources and, simultaneously, to achieve a more complete characterization of different aspects of brain activity during cognitive processing. Owing to its good time resolution and considerably better spatial accuracy than that provided by EEG, magnetoencephalography holds great promise as a tool for revealing information-processing sequences of the human brain.

Human auditory cortex responses to rising versus falling glides

A 24-channel SQUID magnetometer was used to record signals from the right auditory cortex to tone glides with 16 different centre frequencies (ranging from 0.5 to 2 kHz), sweeping over one octave. The stimulus sequence, presented with an interstimulus onset interval of 0.6 s, consisted of infrequent 'deviant' rising (falling) glides and frequent 'standard' falling (rising) glides; the rising glides were identical to the falling glides but presented in the reverse direction. Deviant glides elicited significantly larger responses than standards at a latency of about 100 ms. This amplitude difference presumably arises from a mismatch response to the deviants. These results suggest that the auditory cortex extracts the direction of frequency transition even when the stimuli do not contain same frequencies.