Phonemes, intensity and attention: differential effects on the mismatch negativity (MMN)

Auditory change detection and visual selective attention: association between MMN and N2pc

While the auditory and visual systems each provide distinct information to our brain, they also work together to process and prioritize input to address ever-changing conditions. Previous studies highlighted the trade-off between auditory change detection and visual selective attention; however, the relationship between them is still unclear. Here, we recorded electroencephalography signals from 106 healthy adults in three experiments. Our findings revealed a positive correlation at the population level between the amplitudes of event-related potential indices associated with auditory change detection (mismatch negativity) and visual selective attention (posterior contralateral N2) when elicited in separate tasks. This correlation persisted even when participants performed a visual task while disregarding simultaneous auditory stimuli. Interestingly, as visual attention demand increased, participants whose posterior contralateral N2 amplitude increased the most exhibited the largest reduction in mismatch negativity, suggesting a within-subject trade-off between the two processes. Taken together, our results suggest an intimate relationship and potential shared mechanism between auditory change detection and visual selective attention. We liken this to a total capacity limit that varies between individuals, which could drive correlated individual differences in auditory change detection and visual selective attention, and also within-subject competition between the two, with task-based modulation of visual attention causing within-participant decrease in auditory change detection sensitivity.

Automatic Sensory Predictions: A Review of Predictive Mechanisms in the Brain and Their Link to Conscious Processing

The human brain has the astonishing capacity of integrating streams of sensory information from the environment and forming predictions about future events in an automatic way. Despite being initially developed for visual processing, the bulk of predictive coding research has subsequently focused on auditory processing, with the famous mismatch negativity signal as possibly the most studied signature of a surprise or prediction error (PE) signal. Auditory PEs are present during various consciousness states. Intriguingly, their presence and characteristics have been linked with residual levels of consciousness and return of awareness. In this review we first give an overview of the neural substrates of predictive processes in the auditory modality and their relation to consciousness. Then, we focus on different states of consciousness - wakefulness, sleep, anesthesia, coma, meditation, and hypnosis - and on what mysteries predictive processing has been able to disclose about brain functioning in such states. We review studies investigating how the neural signatures of auditory predictions are modulated by states of reduced or lacking consciousness. As a future outlook, we propose the combination of electrophysiological and computational techniques that will allow investigation of which facets of sensory predictive processes are maintained when consciousness fades away.

Behavioral effect of mismatch negativity neurofeedback on foreign language learning

Listening is critical for foreign language learning. Listening difficulties can occur because of an inability to perceive or recognize sounds while listening to speech, whereas successful listening can boost understanding and improve speaking when learning a foreign language. Previous studies in our laboratory revealed that EEG-neurofeedback (NF) using mismatch negativity event-related brain potential successfully induced unconscious learning in terms of auditory discrimination of speech sounds. Here, we conducted a feasibility study with a small participant group (NF group and control group; six participants each) to examine the practical effects of mismatch negativity NF for improving the perception of speech sounds in a foreign language. Native Japanese speakers completed a task in which they learned to perceive and recognize spoken English words containing the consonants "l" or "r". Participants received neurofeedback training while not explicitly attending to auditory stimuli. The results revealed that NF training significantly improved the proportion of correct in discrimination and recognition trials, even though the training time for each word pair was reduced to 20% of the training time reported in our previous study. The learning effect was not affected by training with three pairs of words with different vowels. The current results indicate that NF resulted in long-term learning that persisted for at least 2 months.

Asymmetries in Accessing Vowel Representations Are Driven by Phonological and Acoustic Properties: Neural and Behavioral Evidence From Natural German Minimal Pairs

In vowel discrimination, commonly found discrimination patterns are directional asymmetries where discrimination is faster (or easier) if differing vowels are presented in a certain sequence compared to the reversed sequence. Different models of speech sound processing try to account for these asymmetries based on either phonetic or phonological properties. In this study, we tested and compared two of those often-discussed models, namely the Featurally Underspecified Lexicon (FUL) model (Lahiri and Reetz, 2002) and the Natural Referent Vowel (NRV) framework (Polka and Bohn, 2011). While most studies presented isolated vowels, we investigated a large stimulus set of German vowels in a more naturalistic setting within minimal pairs. We conducted an mismatch negativity (MMN) study in a passive and a reaction time study in an active oddball paradigm. In both data sets, we found directional asymmetries that can be explained by either phonological or phonetic theories. While behaviorally, the vowel discrimination was based on phonological properties, both tested models failed to explain the found neural patterns comprehensively. Therefore, we additionally examined the influence of a variety of articulatory, acoustical, and lexical factors (e.g., formant structure, intensity, duration, and frequency of occurrence) but also the influence of factors beyond the well-known (perceived loudness of vowels, degree of openness) in depth via multiple regression analyses. The analyses revealed that the perceptual factor of perceived loudness has a greater impact than considered in the literature and should be taken stronger into consideration when analyzing preattentive natural vowel processing.

Making Sense of Mismatch Negativity

Evoked potentials provide valuable insight into brain processes that are integral to our ability to interact effectively and efficiently in the world. The mismatch negativity (MMN) component of the evoked potential has proven highly informative on the ways in which sensitivity to regularity contributes to perception and cognition. This review offers a compendium of research on MMN with a view to scaffolding an appreciation for its use as a tool to explore the way regularities contribute to predictions about the sensory environment over many timescales. In compiling this work, interest in MMN as an index of sensory encoding and memory are addressed, as well as attention. Perspectives on the possible underlying computational processes are reviewed as well as recent observations that invite consideration of how MMN relates to how we learn, what we learn, and why.

Music-induced positive mood broadens the scope of auditory attention

Previous studies indicate that positive mood broadens the scope of visual attention, which can manifest as heightened distractibility. We used event-related potentials (ERP) to investigate whether music-induced positive mood has comparable effects on selective attention in the auditory domain. Subjects listened to experimenter-selected happy, neutral or sad instrumental music and afterwards participated in a dichotic listening task. Distractor sounds in the unattended channel elicited responses related to early sound encoding (N1/MMN) and bottom-up attention capture (P3a) while target sounds in the attended channel elicited a response related to top-down-controlled processing of task-relevant stimuli (P3b). For the subjects in a happy mood, the N1/MMN responses to the distractor sounds were enlarged while the P3b elicited by the target sounds was diminished. Behaviorally, these subjects tended to show heightened error rates on target trials following the distractor sounds. Thus, the ERP and behavioral results indicate that the subjects in a happy mood allocated their attentional resources more diffusely across the attended and the to-be-ignored channels. Therefore, the current study extends previous research on the effects of mood on visual attention and indicates that even unfamiliar instrumental music can broaden the scope of auditory attention via its effects on mood.

Neurophysiological and Behavioral Responses of Mandarin Lexical Tone Processing

Language experience enhances discrimination of speech contrasts at a behavioral- perceptual level, as well as at a pre-attentive level, as indexed by event-related potential (ERP) mismatch negativity (MMN) responses. The enhanced sensitivity could be the result of changes in acoustic resolution and/or long-term memory representations of the relevant information in the auditory cortex. To examine these possibilities, we used a short (ca. 600 ms) vs. long (ca. 2,600 ms) interstimulus interval (ISI) in a passive, oddball discrimination task while obtaining ERPs. These ISI differences were used to test whether cross-linguistic differences in processing Mandarin lexical tone are a function of differences in acoustic resolution and/or differences in long-term memory representations. Bisyllabic nonword tokens that differed in lexical tone categories were presented using a passive listening multiple oddball paradigm. Behavioral discrimination and identification data were also collected. The ERP results revealed robust MMNs to both easy and difficult lexical tone differences for both groups at short ISIs. At long ISIs, there was either no change or an enhanced MMN amplitude for the Mandarin group, but reduced MMN amplitude for the English group. In addition, the Mandarin listeners showed a larger late negativity (LN) discriminative response than the English listeners for lexical tone contrasts in the long ISI condition. Mandarin speakers outperformed English speakers in the behavioral tasks, especially under the long ISI conditions with the more similar lexical tone pair. These results suggest that the acoustic correlates of lexical tone are fairly robust and easily discriminated at short ISIs, when the auditory sensory memory trace is strong. At longer ISIs beyond 2.5 s language-specific experience is necessary for robust discrimination.

Neural measures of a Japanese consonant length discrimination by Japanese and American English listeners: Effects of attention

This study examined automaticity of discrimination of a Japanese length contrast for consonants (miʃi vs. miʃʃi) in native (Japanese) and non-native (American-English) listeners using behavioral measures and the event-related potential (ERP) mismatch negativity (MMN). Attention to the auditory input was manipulated either away from the auditory input via a visual oddball task (Visual Attend), or to the input by asking the listeners to count auditory deviants (Auditory Attend). Results showed a larger MMN when attention was focused on the consonant contrast than away from it for both groups. The MMN was larger for consonant duration increments than decrements. No difference in MMN between the language groups was observed, but the Japanese listeners did show better behavioral discrimination than the American English listeners. In addition, behavioral responses showed a weak, but significant correlation with MMN amplitude. These findings suggest that both acoustic-phonetic properties and phonological experience affects automaticity of speech processing. This article is part of a Special Issue entitled SI: Prediction and Attention.

A hierarchy of responses to auditory regularities in the macaque brain

Can monkeys learn simple auditory sequences and detect when a new sequence deviates from the stored pattern? Here we tested the predictive-coding hypothesis, which postulates that cortical areas encode internal models of sensory sequences at multiple hierarchical levels, and use these predictive models to detect deviant stimuli. In humans, hierarchical predictive coding has been supported by studies of auditory sequence processing, but it is unclear whether internal hierarchical models of auditory sequences are also available to nonhuman animals. Using fMRI, we evaluated the encoding of auditory regularities in awake monkeys listening to first- and second-order sequence violations. We observed distinct fMRI responses to first-order violations in auditory cortex and to second-order violations in a frontoparietal network, a distinction only demonstrated in conscious humans so far. The results indicate that the capacity to represent and predict the structure of auditory sequences is shared by humans and nonhuman primates.

The five myths of MMN: redefining how to use MMN in basic and clinical research

The goal of this review article is to redefine what the mismatch negativity (MMN) component of event-related potentials reflects in auditory scene analysis, and to provide an overview of how the MMN serves as a valuable tool in Cognitive Neuroscience research. In doing so, some of the old beliefs (five common 'myths') about MMN will be dispelled, such as the notion that MMN is a simple feature discriminator and that attention itself modulates MMN elicitation. A revised description of what MMN truly reflects will be provided, which includes a principal focus onto the highly context-dependent nature of MMN elicitation and new terminology to discuss MMN and attention. This revised framework will help clarify what has been a long line of seemingly contradictory results from studies in which behavioral ability to hear differences between sounds and passive elicitation of MMN have been inconsistent. Understanding what MMN is will also benefit clinical research efforts by providing a new picture of how to design appropriate paradigms suited to various clinical populations.

Reduced mismatch negativity predates the onset of psychosis

BACKGROUND

Individuals with an "At Risk Mental State" have a 20-30% chance of developing a psychotic disorder within two years; however it is difficult to predict which individuals will become ill on the basis of their clinical symptoms alone. We examined whether mismatch negativity (MMN) could help to identify those who are particularly likely to make a transition to psychosis.

METHOD

41 cases meeting PACE criteria for the At Risk Mental State (ARMS) and 50 controls performed a duration-deviant passive auditory oddball task whilst their electroencephalogram was recorded. The amplitude of the MMN wave was compared between groups using linear regression. The ARMS subjects were then followed for 2 years to determine their clinical outcome.

RESULTS

The MMN amplitude was significantly reduced in the ARMS group compared to controls. Of the at-risk subjects who completed followed up (n=41), ten (24% of baseline sample) subsequently developed psychosis. The MMN amplitude in this subgroup was significantly smaller across all three recording sites (FZ, F3 and F4) than in the ARMS individuals who did not become psychotic.

CONCLUSION

Among those with the ARMS, MMN amplitude reduction is associated with an increased likelihood of developing frank psychosis.

Native language experience influences the topography of the mismatch negativity to speech

The ability to learn second language speech sound categories declines during development. We examined this phenomenon by studying the mismatch negativity (MMN) to the /r/ - /l/ distinction in native English speakers and learners of English as a second language who are native speakers of Japanese. Previous studies have suggested that the MMN is remarkably plastic when evaluated as a waveform at a central electrode. We replicated this finding: analyses of the MMN at a typical electrode location (Fz) revealed only small, non-significant differences between groups, despite large behavioral differences in the ability to discriminate these sounds from one another. Topographic analyses, however, revealed reliable differences in lateralization of the MMN, such that native English speakers' responses were left-lateralized relative to native Japanese speakers' responses.

Pre-attentive and attentive processing of French vowels

This study aimed at investigating the effects of acoustic distance and of speaker variability on the pre-attentive and attentive perception of French vowels by French adult speakers. The electroencephalogram (EEG) was recorded while participants watched a silent movie (Passive condition) and discriminated deviant vowels (Active condition). The auditory sequence included 4 French vowels, /u/ (standard) and /o/, /y/ and /ø/ as deviants, produced by 3 different speakers. As the vowel /o/ is closer to /u/ than the other deviants in acoustic distance, we predicted smaller mismatch negativity (MMN) and smaller N1 component, as well as higher error rate and longer reaction times. Results were in line with these predictions. Moreover, the MMN was elicited by all deviant vowels independently of speaker variability. By contrast, the Vowel by Speaker interaction was significant in the Active listening condition thereby showing that subtle within-category differences are processed at the attentive level. These results suggest that while vowels are categorized pre-attentively according to phonemic representations and independently of speaker variability, participants are sensitive to between-speaker differences when they focus attention on vowel processing.

Perception of a Japanese vowel length contrast by Japanese and American English listeners: behavioral and electrophysiological measures

This study examined the role of automatic selective perceptual processes in native and non-native listeners' perception of a Japanese vowel length contrast (tado vs. taado), using multiple, natural-speech tokens of each category as stimuli in a "categorial oddball" design. Mismatch negativity (MMN) was used to index discrimination of the temporally-cued vowel contrast by naïve adult American listeners and by a native Japanese-speaking control group in two experiments in which attention to the auditory input was manipulated: in Exp 1 (Visual-Attend), listeners silently counted deviants in a simultaneously-presented visual categorial oddball shape discrimination task; in Exp 2 (Auditory-Attend), listeners attended to the auditory input and implicitly counted target deviants. MMN results showed effects of language experience and attentional focus: MMN amplitudes were smaller for American compared to Japanese listeners in the Visual-Attend Condition and for the American listeners in the Visual compared to Auditory-Attend Condition. Subtle differences in topography were also seen, specifically in that the Japanese group showed more robust responses than the American listeners at left hemisphere scalp sites that probably index activity from the superior temporal gyrus. Follow-up behavioral discrimination tests showed that Americans discriminated the contrast well above chance, but more poorly than did Japanese listeners. This pattern of electrophysiological and behavioral results supports the conclusion that early experience with phonetic contrasts of a language results in changes in neural representations in the auditory cortex that allow for more robust automatic, phonetic processing of native-language speech input.

A neuroanatomically grounded Hebbian-learning model of attention-language interactions in the human brain

Meaningful familiar stimuli and senseless unknown materials lead to different patterns of brain activation. A late major neurophysiological response indexing ‘sense’ is the negative component of event-related potential peaking at around 400 ms (N400), an event-related potential that emerges in attention-demanding tasks and is larger for senseless materials (e.g. meaningless pseudowords) than for matched meaningful stimuli (words). However, the mismatch negativity (latency 100–250 ms), an early automatic brain response elicited under distraction, is larger to words than to pseudowords, thus exhibiting the opposite pattern to that seen for the N400. So far, no theoretical account has been able to reconcile and explain these findings by means of a single, mechanistic neural model. We implemented a neuroanatomically grounded neural network model of the left perisylvian language cortex and simulated: (i) brain processes of early language acquisition and (ii) cortical responses to familiar word and senseless pseudoword stimuli. We found that variation of the area-specific inhibition (the model correlate of attention) modulated the simulated brain response to words and pseudowords, producing either an N400- or a mismatch negativity-like response depending on the amount of inhibition (i.e. available attentional resources). Our model: (i) provides a unifying explanatory account, at cortical level, of experimental observations that, so far, had not been given a coherent interpretation within a single framework; (ii) demonstrates the viability of purely Hebbian, associative learning in a multilayered neural network architecture; and (iii) makes clear predictions on the effects of attention on latency and magnitude of event-related potentials to lexical items. Such predictions have been confirmed by recent experimental evidence.

Syntax as a reflex: neurophysiological evidence for early automaticity of grammatical processing

It has been a matter of debate whether the specifically human capacity to process syntactic information draws on attentional resources or is automatic. To address this issue, we recorded neurophysiological indicators of syntactic processing to spoken sentences while subjects were distracted to different degrees from language processing. Subjects were either passively distracted, by watching a silent video film, or their attention was actively streamed away from the language input by performing a demanding acoustic signal detection task. An early index of syntactic violations, the syntactic Mismatch Negativity (sMMN), distinguished between grammatical and ungrammatical speech even under strongest distraction. The magnitude of the early sMMN (at <150ms) was unaffected by attention load of the distraction task. The independence of the early syntactic brain response of attentional distraction provides neurophysiological evidence for the automaticity of syntax and for its autonomy from other attention-demanding processes, including acoustic stimulus discrimination. The first attentional modulation of syntactic brain responses became manifest at a later stage, at approximately 200ms, thus demonstrating the narrowness of the early time window of syntactic autonomy. We discuss these results in the light of modular and interactive theories of cognitive processing and draw inferences on the automaticity of both the cognitive MMN response and certain grammar processes in general.

The mismatch negativity (MMN) in basic research of central auditory processing: a review

In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.

A New View on the MMN and Attention Debate

The question of whether the mismatch negativity (MMN) is modulated by attention has been debated for over a decade. Although the MMN is widely regarded as reflecting a preattentive auditory process, many studies have shown attention effects on MMN. So, what does preattentive mean if attention can modulate the MMN? To understand the function of MMN in auditory processing, we need to shed new light on the “MMN and attention” debate. This review will discuss the apparent paradox that MMN can be modulated by attention and still be considered an attention-independent process, and provide a new framework for viewing the MMN system. The new model proposes that the principal factor governing MMN is the sound context. MMN generation relies on multiple processing mechanisms that are part of a larger system of auditory scene analysis.

Direct evidence for differential roles of temporal and frontal components of auditory change detection

Automatic change detection is a fundamental capacity of the human brain. In audition, this capacity is indexed by the mismatch negativity (MMN) event-related potential, which is putatively supported by a network consisting of superior temporal and frontal nodes. The aim of this study was to elucidate the roles of these nodes within the neural network of change detection. We used a dichotic paradigm in which subjects (N=14) attended targets defined by either pitch or spatial location in one auditory stream while the MMN was measured in response to unattended deviants of pitch and spatial location in the other stream. The frontal and temporal components of the MMN were examined using current source density (CSD) measurements. Competition for processing resources nearly eliminated the temporal CSD mismatch response, in a highly feature-specific manner: the response to spatial location deviants was reduced when the target dimension was spatial location but not when it was pitch, whereas the reverse occurred for pitch deviants. In contrast, the frontal CSD mismatch response was neither affected by competition nor by general attention demands. Thus, within the network of change detection, the temporal generators are specifically associated with processing feature-specific information, whereas the role of the frontal generators remains unclear. Moreover, the results are inconsistent with a serial model in which the frontal generator is contingent on activation of the temporal generator.

The effect of visual task difficulty and attentional direction on the detection of acoustic change as indexed by the Mismatch Negativity

Näätänen's model of auditory processing purports that attention does not affect the MMN. The present study investigates this claim through two different manipulations. First, the effect of visual task difficulty on the passively elicited MMN is assessed. Second, the MMNs elicited by stimuli under attended and ignored conditions are compared. In Experiment 1, subjects were presented with mixed sequences of equiprobable auditory and visual stimuli. The auditory stimuli consisted of standard (80 dB SPL 1000 Hz), frequency deviant (1050 Hz), and intensity deviant (70 dB SPL) tone pips. In a first instance, subjects were instructed to ignore the auditory stimulation and engage in an easy and difficult visual discrimination task (focused condition). Subsequently, they were asked to attend to both modalities and detect visual and auditory deviant stimuli (divided condition). The results indicate that the passively elicited MMN to frequency and intensity deviants did not significantly vary with visual task difficulty, in spite of the fact that the easy and difficult tasks showed a wide variation in performance. The manipulation of the attentional direction (focused vs. divided conditions) did result in a significant effect on the MMN elicited by the intensity, but not frequency, deviant. The intensity MMN was larger at frontal sites when subjects' attention was directed to both modalities as compared to only the visual modality. The attentional effect on the MMN to the intensity deviants only may be due to the specific deviant feature or the poorer perceptual discriminability of this deviant from the standard. Experiment 2 was designed to address this issue. The methods of Experiment 2 were identical to those of Experiment 1 with the exception that the intensity deviant (60 dB SPL) was made to be more perceptible than the frequency deviant (1016 Hz) when compared to the standard stimulus (80 dB SPL 1000 Hz). The results of Experiment 2 also demonstrated that the passively elicited MMN was not affected by large variations in visual task difficulty; this provides convincing evidence that the MMN is independent of visual task demands. Similarly to Experiment 1, the direction of attention again had a significant effect on the MMN. In Experiment 2, however, the frequency MMN (and not the intensity MMN) was larger at frontal sites during divided attention compared to focused visual attention. The most parsimonious explanation of these results is that attention enhances the discriminability of the deviant from the standard background stimulation. As such, small acoustic changes would benefit from attention whereas the discriminability of larger changes may not be significantly enhanced.

“…and were instructed to read a self-selected book while ignoring the auditory stimuli”: The effects of task demands on the mismatch negativity

OBJECTIVE

The Mismatch Negativity (MMN) is commonly recorded while the subject is reading, and instructed to ignore the auditory stimuli. It is generally assumed that the demands of the diversion task will have no effect on the MMN. Several studies, however, have reported that a diversion task presumably requiring strong attentional focus is associated with a smaller MMN than that elicited during a less demanding task. This study examines the effect of variations in the classical reading paradigm on the MMN.

METHODS

In Experiment 1, event-related potentials (ERP) were recorded while subjects were presented with standard (80 dB SPL 1000 Hz) and frequency deviant (1050 Hz) stimuli. Subjects were instructed to ignore the tone pips and, in separate conditions, engage in different tasks. They were asked to read a text or to sit passively. Subjects were informed that they would subsequently be queried or not about the content of the reading. In Experiment 2, the auditory sequence included the same standard (80 dB SPL 1000 Hz) but the deviant was changed to an intensity decrement (70 dB SPL). A different sample of subjects was again asked to ignore the auditory stimuli and engage in different reading tasks that would or not be followed by query.

RESULTS

In all task conditions, MMN was elicited by the frequency and intensity change. The intensity MMN did not significantly vary with task. A significant effect of task was, however, found for the frequency MMN. Its amplitude was largest when subjects were later queried about their reading.

CONCLUSIONS

This finding is counter-intuitive in light of previous research on the attentional modulation of the MMN. The pattern of frequency MMN results may relate to the differences in cortical excitability across tasks.

SIGNIFICANCE

The present results indicate that the nature of the diversion task may affect the MMN. The choice of diversion task during MMN recording should thus be carefully considered.

Determinants of dominance: Is language laterality explained by physical or linguistic features of speech?

The nature of cerebral asymmetry of the language function is still not fully understood. Two main views are that laterality is best explained (1) by left cortical specialization for the processing of spectrally rich and rapidly changing sounds, and (2) by a predisposition of one hemisphere to develop a module for phonemes. We tested both of these views by investigating magnetic brain responses to the same brief acoustic stimulus, placed in contexts where it was perceived either as a noise burst with no resemblance of speech, or as a native language sound being part of a meaningless pseudoword. In further experiments, the same acoustic element was placed in the context of words. We found reliable left hemispheric dominance only when the sound was placed in word context. These results, obtained in a passive odd-ball paradigm, suggest that neither physical properties nor phoneme status of a sound are sufficient for laterality. In order to elicit left lateralized cortical activation in normal right-handed individuals, a rapidly changing spectrally rich sound with phoneme status needs to be placed in the context of frequently encountered larger language elements, such as words. This demonstrates that language laterality is bound to the processing of sounds as units of frequently occurring meaningful items and can thus be linked to the processes of learning and memory trace formation for such items rather than to their physical or phonological properties.

Effects of language experience: Neural commitment to language-specific auditory patterns

Linguistic experience alters an individual's perception of speech. We here provide evidence of the effects of language experience at the neural level from two magnetoencephalography (MEG) studies that compare adult American and Japanese listeners' phonetic processing. The experimental stimuli were American English /ra/ and /la/ syllables, phonemic in English but not in Japanese. In Experiment 1, the control stimuli were /ba/ and /wa/ syllables, phonemic in both languages; in Experiment 2, they were non-speech replicas of /ra/ and /la/. The behavioral and neuromagnetic results showed that Japanese listeners were less sensitive to the phonemic /r-l/ difference than American listeners. Furthermore, processing non-native speech sounds recruited significantly greater brain resources in both hemispheres and required a significantly longer period of brain activation in two regions, the superior temporal area and the inferior parietal area. The control stimuli showed no significant differences except that the duration effect in the superior temporal cortex also applied to the non-speech replicas. We argue that early exposure to a particular language produces a "neural commitment" to the acoustic properties of that language and that this neural commitment interferes with foreign language processing, making it less efficient.

Automatic and controlled processing of acoustic and phonetic contrasts

Changes in the temporal properties of the speech signal provide important cues for phoneme identification. An impairment or inability to detect such changes may adversely affect one's ability to understand spoken speech. The difference in meaning between the Finnish words tuli (fire) and tuuli (wind), for example, lies in the difference between the duration of the vowel /u/. Detecting changes in the temporal properties of the speech signal, therefore, is critical for distinguishing between phonemes and identifying words. In the current study, we tested whether detection of changes in speech sounds, in native Finnish speakers, would vary as a function of the position within the word that the informational changes occurred (beginning, middle, or end) by evaluating how length contrasts in segments of three-syllable Finnish pseudo-words and their acoustic correlates were discriminated. We recorded a combination of cortical components of event-related brain potentials (MMN, N2b, P3b) along with behavioral measures of the perception of the same sounds. It was found that speech sounds were not processed differently than non-speech sounds in the early stages of auditory processing indexed by MMN. Differences occurred only in later stages associated with controlled processes. The effects of position and attention on speech and non-speech stimuli are discussed.

Language-specific memory traces of consonants in the brain

This study examined whether experience with a native language affected processing of a place-of-articulation feature. In Experiment 1, 10 stimuli from a continuum of synthesized Hindi bilabial, dental and retroflexed stops were presented to English and Hindi speakers to examine discrimination and identification across the bilabial/dental and dental/retroflexed boundaries. In an oddball task designed to elicit mismatch negativity (MMN), subjects ignored these stimuli while their brain activity was recorded. All participants showed similar behavioral discrimination and identification. However, the English subjects were slower that the Hindi subjects on the discrimination task. All subjects were less accurate and slower at identifying and discriminating the dental and retroflexed compared to the bilabial sounds. A small MMN was observed for some of the bilabial-dental contrasts, but not for the dental-retroflexed contrasts. No group differences were found. In Experiment 2, MMN was observed to a greater stimulus difference (bilabial-retroflexed) and was earlier when the bilabial rather than the retroflexed sound served as the frequent stimulus for both groups. The MMN was also earlier for the Hindi than the English groups when the retroflexed sound served as the frequent stimulus. These results indicate that the Hindi speakers used detailed acoustic-phonetic information for more rapid brain discrimination than the English participants and that the dental-retroflexed discrimination is more difficult than the bilabial-dental discrimination for all speakers.

Language context and phonetic change detection

Event-related brain potentials (ERP) were recorded to two spoken words, /paeti/ and /peti/. The vowel difference between the two words results in a semantical difference in Finnish, but not in Hungarian, in which /ae/ and /e/ are perceived as allophones of the same vowel /epsilon/. As a consequence, native Hungarian speakers, who had not studied Finnish before being tested, could not categorize the two word stimuli. In the main experiment, native Hungarian speakers, who fluently spoke Finnish, were presented with two oddball sequences in which /paeti/ was the frequent standard stimulus, /peti/ the infrequent deviant. In addition, very rare target words were also included. In one condition, the targets were Hungarian words, whereas in the other, they were Finnish words. The participants' sense of being in two different language environments was further encouraged by having separate experimenters conducting the two conditions, one speaking with the subjects only in Hungarian, the other only in Finnish. Language context had no effect on the mismatch negativity ERP component elicited by the deviant word stimuli. This result suggests that language context does not affect the pre-attentive detection of auditory deviance.

Processing of dynamic aspects of speech and non-speech stimuli: a whole-head magnetoencephalography study

Clinical and experimental data indicate higher proficiency of the left hemisphere in encoding dynamic acoustic events such as rapid formant transitions (30-40 ms) that distinguish consonant-vowel syllables such as /ba/ or /da/. In order to further elucidate the underlying neurophysiological mechanisms, discrimination of /bi/-like formant transitions of variable duration (18, 36, 54, or 72 ms) from a steady-state /i/-like vowel was investigated by means of whole-head magnetoencephalography (MEG) both during visual distraction and selective attention. Voiced speech-like as well as unvoiced non-speech stimuli, matched for spectral envelope, served as test materials. Based on an oddball design, magnetic mismatch fields (MMF) were determined during an early (170-210 ms) and a late (230-290 ms) time window. Selective attention toward the deviant events resulted in enhanced MMFs particularly within the left hemisphere, indicating attention-dependent left-lateralized processing of dynamic auditory events across both the speech and non-speech domains. Perceptual discrimination improved along with transient lengthening. Accordingly, early MMF was, as a rule, enlarged in case of longer as compared to shorter transients. The 36-ms transitions yielded attention- and voicing-dependent deviations from the linear regression of MMF strength on transition duration. Considering the predominance of 30- to 40-ms formant transients across the world's languages, these findings indicate an adaptation or predisposition of the human perceptual system to the spectral/temporal characteristics of prototypical speech sounds. Signal voicing had no significant main effect on MMF strength despite superior perceptual performance in case of voiced as compared to voiceless target stimuli.

MMN and attention: competition for deviance detection

We addressed the question of whether the mismatch negativity (MMN) event-related potential reflects an attention-independent process. Previous studies have shown that the MMN response to intensity deviation was significantly reduced or even abolished when attention was highly focused on a concurrent sound channel, whereas no conclusive evidence of attentional sensitivity has been obtained for frequency deviation. We tested a new hypothesis suggesting that competition between detection of identical deviations in attended and unattended channels and the biasing of this competition induced by the subject's task account for the observed MMN effects. In a fast-paced dichotic paradigm, we set up competition for frequency MMN and removed it for intensity MMN. We found that frequency MMN was now abolished in the unattended channel, whereas the amplitude of the intensity MMN was unaffected. These results support the competition hypothesis and suggest that selective attention in and of itself does not affect the MMN. Top-down processes can determine what information reaches the deviance-detection process when changes in multiple channels vie for the same MMN resource and one of the competing changes is relevant for the subject's task.

Temporal integration: intentional sound discrimination does not modulate stimulus-driven processes in auditory event synthesis

OBJECTIVE

Our previous study showed that the auditory context could influence whether two successive acoustic changes occurring within the temporal integration window (approximately 200ms) were pre-attentively encoded as a single auditory event or as two discrete events (Cogn Brain Res 12 (2001) 431). The aim of the current study was to assess whether top-down processes could influence the stimulus-driven processes in determining what constitutes an auditory event.

METHODS

Electroencepholagram (EEG) was recorded from 11 scalp electrodes to frequently occurring standard and infrequently occurring deviant sounds. Within the stimulus blocks, deviants either occurred only in pairs (successive feature changes) or both singly and in pairs. Event-related potential indices of change and target detection, the mismatch negativity (MMN) and the N2b component, respectively, were compared with the simultaneously measured performance in discriminating the deviants.

RESULTS

Even though subjects could voluntarily distinguish the two successive auditory feature changes from each other, which was also indicated by the elicitation of the N2b target-detection response, top-down processes did not modify the event organization reflected by the MMN response.

CONCLUSIONS

Top-down processes can extract elemental auditory information from a single integrated acoustic event, but the extraction occurs at a later processing stage than the one whose outcome is indexed by MMN.

SIGNIFICANCE

Initial processes of auditory event-formation are fully governed by the context within which the sounds occur. Perception of the deviants as two separate sound events (the top-down effects) did not change the initial neural representation of the same deviants as one event (indexed by the MMN), without a corresponding change in the stimulus-driven sound organization.

Hemispheric lateralization of the processing of consonant-vowel syllables (formant transitions): effects of stimulus characteristics and attentional demands on evoked magnetic fields

It is still unsettled in how far temporal resolution of dynamic acoustic events (formant transitions) or phonetic/linguistic processes contribute to predominant left-hemisphere encoding of consonant-vowel syllables. To further elucidate the underlying mechanisms, evoked magnetic fields in response to consonant-vowel events (synthetic versus spoken) were recorded (oddball design: standards=binaural/ba/, deviants=dichotic/ba/-/da/; 20 right-handed subjects) under different attentional conditions (visual distraction versus stimulus identification). Spoken events yielded a left-lateralized peak phase of the mismatch field (MMF; 150-200ms post-stimulus onset) in response to right-ear deviants during distraction. By contrast, pre-attentive processing of synthetic items gave rise to a left-enhanced MMF onset (100ms), but failed to elicit later lateralization effects. In case of directed attention, synthetic deviants elicited a left-pronounced MMF peak resembling the pre-attentive response to natural syllables. These interactions of MMF asymmetry with signal structure and attentional load indicate two distinct successive left-lateralization effects: signal-related operations and representation of 'phonetic traces'. Furthermore, a right-lateralized early MMF component (100ms) emerged in response to natural syllables during pre-attentive processing and to synthetic stimuli in case of directed attention. Conceivably, these effects indicate right hemisphere operations prior to phonetic evaluation such as periodicity representation. Two distinct time windows showed correlations between dichotic listening performance and ear effects on magnetic responses reflecting early gain factors (ca. 75ms post-stimulus onset) and binaural fusion strategies (ca. 200ms), respectively. Finally, gender interacted with MMF lateralization, indicating different processing strategies in case of artificial speech signals.

Listening to polyphonic music recruits domain-general attention and working memory circuits

Polyphonic music combines multiple auditory streams to create complex auditory scenes, thus providing a tool for investigating the neural mechanisms that orient attention in natural auditory contexts. Across two fMRI experiments, we varied stimuli and task demands in order to identify the cortical areas that are activated during attentive listening to real music. In individual experiments and in a conjunction analysis of the two experiments, we found bilateral blood oxygen level dependent (BOLD) signal increases in temporal (the superior temporal gyrus), parietal (the intraparietal sulcus), and frontal (the precentral sulcus, the inferior frontal sulcus and gyrus, and the frontal operculum) areas during selective and global listening, as compared with passive rest without musical stimulation. Direct comparisons of the listening conditions showed significant differences between attending to single timbres (instruments) and attending across multiple instruments, although the patterns that were observed depended on the relative demands of the tasks being compared. The overall pattern of BOLD signal increases indicated that attentive listening to music recruits neural circuits underlying multiple forms of working memory, attention, semantic processing, target detection, and motor imagery. Thus, attentive listening to music appears to be enabled by areas that serve general functions, rather than by music-specific cortical modules.

Asymmetrical mismatch negativity in humans as determined by phonetic but not physical difference

A two-tone oddball procedure was employed to examine the effect of a phonemic category on the mismatch negativity (MMN). One of the stimuli was a phoneme prototype of Japanese /e/, and the other, [e/ö], which was perceived by Japanese participants as showing deviance from typicality but is nonetheless included in the category /e/. As control stimuli, a pair of pure tones (1940 and 1794 Hz), corresponding to the F2 frequencies of /e/ and [e/ö], respectively, was presented within the same oddball procedure. The MMN for deviant [e/ö] revealed greater amplitude than that of deviant /e/, although there was no significant difference in amplitude between the pure tones. The results suggest that a phonemic category determines the auditory sensory memory.

The neurophysiology of auditory perception: from single units to evoked potentials

Evoked electric potential and magnetic field studies have the immense benefit that they can be conducted in awake, behaving humans and can be directly correlated with aspects of perception. As such, they are powerful objective indicators of perceptual properties. However, given a set of evoked potential and/or evoked field waveforms and their source locations, obtained for an exhaustive set of stimuli and stimulus contrasts, is it possible to determine blindly, i.e. predict, what the stimuli or stimulus contrasts were? If this can be done with some success, then a useful amount of information resides in scalp-recorded activity for, e.g., the study of auditory speech processing. In this review, we compare neural representations based on single-unit and evoked response activity for vowels and consonant-vowel phonemes with distinctions in formant glides and voice onset time. We conclude that temporal aspects of evoked responses can track some of the dominant response features present in single-unit activity. However, N1 morphology does not reliably predict phonetic identification of stimuli varying in voice onset time, and the reported appearance of a double-peak onset response in aggregate recordings from the auditory cortex does not indicate a cortical correlate of the perception of voicelessness. This suggests that temporal aspects of single-unit population activity are likely not inclusive enough for representation of categorical perception boundaries. In contrast to population activity based on single-unit recording, the ability to accurately localize the sources of scalp-evoked activity is one of the bottlenecks in obtaining an accessible neurophysiological substrate of perception. Attaining this is one of the requisites to arrive at the prospect of blind determination of stimuli on the basis of evoked responses. At the current sophistication level of recording and analysis, evoked responses remain in the realm of extremely sensitive objective indicators of stimulus change or stimulus differences. As such, they are signs of perceptual activity, but not comprehensive representations thereof.

Effects of water on cortical excitability in humans

The effects of water on cortical excitability, measured using magnetoencephalographic recordings, were investigated in a sample of 19 healthy volunteers in a double-blind, placebo experiment comparing water with saline solution. Spontaneous magnetoencephalogram as well as auditory-evoked magnetic fields were recorded before and after the drinking of 750 mL water (9 subjects) or saline solution (10 subjects) and during and after hyperventilation following the drinking conditions. Hyperventilation was used to enhance the hypothesized synchronizing effect of water on spontaneous magnetoencephalographic activity. In addition, the magnetic fields were measured during a dichotic listening task under attended and unattended conditions. The prediction, that intake of water, because of induced cell swelling, will increase neuronal excitability and lead to an increased synchronization of the spontaneous magnetoencephalogram during hyperventilation was confirmed. Hyperventilation induced an increase of spectral power in all frequency bands particularly theta and delta power after water drinking. Furthermore, there was an increase of magnetic mismatch negativity (MMNm) amplitude in attended conditions and a simultaneous decrease in unattended conditions after water drinking. N1m (magnetic N1 wave) revealed significant changes during experimental conditions: increase after drinking and decrease after hyperventilation in both groups. MMNm for attended conditions showed a high positive correlation with osmolality changes (difference in the mol solute per kg water before and after drinking); N1m and PNm (magnetic processing negativity) as well as MMNm for unattended conditions showed significant correlations with subjective ratings of thirst and mood state.

Plastic neural changes and reading improvement caused by audiovisual training in reading-impaired children

This study aimed at determining whether audiovisual training without linguistic material has a remediating effect on reading skills and central auditory processing in dyslexic children. It was found that this training resulted in plastic changes in the auditory cortex, indexed by enhanced electrophysiological mismatch negativity and faster reaction times to sound changes. Importantly, these changes were accompanied by improvement in reading skills. The results indicate that reading difficulties can be ameliorated by special training programs and, further, that the training effects can be observed in brain activity. Moreover, the fact that the present training effects were obtained by using a program including no linguistic material indicates that dyslexia is at least partly based on a general auditory perceptual deficit.